NYCPlanning / NYCPlanning/data-engineering
ZTL - make sure comparison is to previous version
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 43
- Forks
- 3
- Avg merge
- 23h 3m
- Merged PRs (30d)
- 44
Description
There's a couple ways to go about this, but we've had a few times now were GIS builds ZTL, but the previous version was never archived with ingest (or even published from draft to publish often, as was the case this time). This often doesn't come up until they've already done some QA, so this has been a big waste of time for them.
There are a few things that would work/help. Worth brainstorming a little as to what feels most appropriate (and maybe check with GIS as well)
- a data check - check that the "previous" version (latest from edm-recipes) is only one month less than the current version
- pull from
edm-publishinglatest instead ofingest(gets rid of a step, though in the case - make the versions much more apparent in the qa files (filenames?)
- make sure that GIS only publishes ZTL from
publishfolder rather thandraftas it seems that they do now (since last month was neverpublished - add something of job to the ztl dataloading action that
- makes sure previous draft was actually published
- archives ztl from publish-latest into edm-recipes
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by tracing the ZTL dataloading action and how it uses edm-recipes, edm-publishing, and the draft and publish folders. Compare the proposed data check, source change, QA filenames, and archive steps with the current workflow; done means an agreed approach prevents QA from proceeding against an unavailable or incorrect previous version.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- data-engineering
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100