NYCPlanning / NYCPlanning/data-engineering
Hudson Square geometry correction is patching stale KPDB input, not ZAP
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 43
- Forks
- 3
- Avg merge
- 23h 3m
- Merged PRs (30d)
- 44
Description
dcp_application.sql swaps in a corrected geometry for the Hudson Square project P2012M0255 from a hand-supplied shapefile. The commit that added it (#2095) says it's needed "until we use the latest ZAP data". That turns out not to be the cause, so bumping ZAP won't retire it.
Where the bad geometry comes from
geom_kpdb in sql/dcp_application.sql prefers dcp_knownprojects over the PLUTO-derived geometry whenever it has one. dcp_knownprojects is KPDB_20240710.zip, a hand-supplied file in edm-private/dcp_housing_team/db-knownprojects/20250430/raw/. It's last year's KPDB, used as an input to this year's.
For P2012M0255 it holds a polygon of 2,115,709 m² whose centroid sits at 40.7817 N, roughly 6 km north of Hudson Square. ZAP is never consulted for this record's geometry.
PLUTO can't fill in either: dcp_projectbbls has zero BBLs for the project, so that path produces nothing.
The correction, HudsonSquare_P2012M0255_4326.zip from the same folder, is 204,591 m² and its centroid lands in MN0201 SoHo-Little Italy-Hudson Square.
Why it matters
With the old geometry the centroid falls outside every NTA, since NTA boundaries exclude water and community district boundaries don't. The project's 3,323 units were then placed by the fallback rather than by location.
What would actually retire the workaround
- a refreshed
dcp_knownprojectswith the right shape, which is the direct fix - ZAP carrying BBLs for the project, so the PLUTO path produces something
- changing the precedence in
geom_kpdbso a staledcp_knownprojectsgeometry doesn't override better sources
Note the folder is named 20250430 but the file inside is KPDB_20240710. The directory is a batch label, not the vintage of its contents.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in sql/dcp_application.sql, tracing geom_kpdb and the dcp_knownprojects and dcp_projectbbls inputs. Inspect KPDB_20240710.zip and HudsonSquare_P2012M0255_4326.zip in edm-private/dcp_housing_team/db-knownprojects/20250430/raw/. Done means the project uses a valid geometry source and its units are assigned to the correct NTA, with the chosen workaround documented or removed.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- sql
- Domain
- data-engineering, databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100