NYCPlanning / NYCPlanning/data-engineering

Tweak `dcpy` folder structure

Open
#1,153 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

platform
Dominant language
Python
Stars
43
Forks
3
Avg merge
23h 3m
Merged PRs (30d)
44

Description

We've discussed that it would be nice to tweak dcpy structure just a bit to make our repo a tad cleaner.

There are two main accepted structures for the layout of a package - src layout and flat layout. Right now, we're effectively doing a flat layout in the root folder of our repo. This isn't quite ideal - mainly, we don't like that pyproject.toml is serving many different purposes right now, and it'd be nice to have all the packaging/dependency information for dcpy within its own folder.

We briefly discussed having something like

dcpy
- pyproject.toml
- src
  - models
  - ...
- test

However, this has one main issue: having dcpy as both a top-level folder and the name of the package messes up pythons import search order - local folder takes priority, and now dcpy doesn't have a models submodule based solely on the folder structure

To solve this, we could rename the top-level folder - something sort of inelegant like dcpy_pkg would work. Or we could also go more the direction that the setuptools doc suggests, that you have one top-level "packages" folder that has packages within (either in src or flat layout). I still honestly don't quite love this - I sort of like this simplicity of one folder per package, with everything needed inside (as opposed to multiple packages with their own source folders, test folders, and all sharing one pyproject.toml).

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by inspecting the repository's root pyproject.toml and current package and test folders, then read the linked setuptools src-layout and flat-layout guidance. Compare the proposed dcpy/src and dcpy/test arrangement with Python's import search-order concern. Done means the repository has an agreed package layout, packaging information is organized consistently, and existing imports and tests still work.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
build-system
Issue type
Refactor
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.