NYCPlanning / NYCPlanning/data-engineering
Tweak `dcpy` folder structure
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 43
- Forks
- 3
- Avg merge
- 23h 3m
- Merged PRs (30d)
- 44
Description
We've discussed that it would be nice to tweak dcpy structure just a bit to make our repo a tad cleaner.
There are two main accepted structures for the layout of a package - src layout and flat layout. Right now, we're effectively doing a flat layout in the root folder of our repo. This isn't quite ideal - mainly, we don't like that pyproject.toml is serving many different purposes right now, and it'd be nice to have all the packaging/dependency information for dcpy within its own folder.
We briefly discussed having something like
dcpy
- pyproject.toml
- src
- models
- ...
- test
However, this has one main issue: having dcpy as both a top-level folder and the name of the package messes up pythons import search order - local folder takes priority, and now dcpy doesn't have a models submodule based solely on the folder structure
To solve this, we could rename the top-level folder - something sort of inelegant like dcpy_pkg would work. Or we could also go more the direction that the setuptools doc suggests, that you have one top-level "packages" folder that has packages within (either in src or flat layout). I still honestly don't quite love this - I sort of like this simplicity of one folder per package, with everything needed inside (as opposed to multiple packages with their own source folders, test folders, and all sharing one pyproject.toml).
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by inspecting the repository's root pyproject.toml and current package and test folders, then read the linked setuptools src-layout and flat-layout guidance. Compare the proposed dcpy/src and dcpy/test arrangement with Python's import search-order concern. Done means the repository has an agreed package layout, packaging information is organized consistently, and existing imports and tests still work.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- build-system
- Issue type
- Refactor
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100