NYCPlanning / NYCPlanning/data-engineering
CSCL - Architecture
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 43
- Forks
- 3
- Avg merge
- 23h 3m
- Merged PRs (30d)
- 44
Description
This pipeline is a bit different from our other build pipelines for a few reasons, the primary being that we are not the main users of it, and the users have some specific requirements. Among other requirements, they need
- A UI that is not just github/azure/orchestration
- ETL to start directly from source - we're not building from
recipesbut from an OracleDB that might have been updated just a few minutes before the pipeline is run
Main complications honestly are around the latter. I'll leave the details for the child issues
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
No files, tests, or entry points are named. Start by reading the child issues referenced in the description and identify how the UI and direct-from-OracleDB ETL requirements are divided among them. Done would require an agreed architecture that addresses those requirements.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- data-engineering
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 20/100