NYCPlanning / NYCPlanning/data-engineering

Hudson Square geometry correction is patching stale KPDB input, not ZAP

Open
#2,590 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

db-knownprojects db-zap
Dominant language
Python
Stars
43
Forks
3
Avg merge
23h 3m
Merged PRs (30d)
44

Description

dcp_application.sql swaps in a corrected geometry for the Hudson Square project P2012M0255 from a hand-supplied shapefile. The commit that added it (#2095) says it's needed "until we use the latest ZAP data". That turns out not to be the cause, so bumping ZAP won't retire it.

Where the bad geometry comes from

geom_kpdb in sql/dcp_application.sql prefers dcp_knownprojects over the PLUTO-derived geometry whenever it has one. dcp_knownprojects is KPDB_20240710.zip, a hand-supplied file in edm-private/dcp_housing_team/db-knownprojects/20250430/raw/. It's last year's KPDB, used as an input to this year's.

For P2012M0255 it holds a polygon of 2,115,709 m² whose centroid sits at 40.7817 N, roughly 6 km north of Hudson Square. ZAP is never consulted for this record's geometry.

PLUTO can't fill in either: dcp_projectbbls has zero BBLs for the project, so that path produces nothing.

The correction, HudsonSquare_P2012M0255_4326.zip from the same folder, is 204,591 m² and its centroid lands in MN0201 SoHo-Little Italy-Hudson Square.

Why it matters

With the old geometry the centroid falls outside every NTA, since NTA boundaries exclude water and community district boundaries don't. The project's 3,323 units were then placed by the fallback rather than by location.

What would actually retire the workaround

  • a refreshed dcp_knownprojects with the right shape, which is the direct fix
  • ZAP carrying BBLs for the project, so the PLUTO path produces something
  • changing the precedence in geom_kpdb so a stale dcp_knownprojects geometry doesn't override better sources

Note the folder is named 20250430 but the file inside is KPDB_20240710. The directory is a batch label, not the vintage of its contents.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in sql/dcp_application.sql, tracing geom_kpdb and the dcp_knownprojects and dcp_projectbbls inputs. Inspect KPDB_20240710.zip and HudsonSquare_P2012M0255_4326.zip in edm-private/dcp_housing_team/db-knownprojects/20250430/raw/. Done means the project uses a valid geometry source and its units are assigned to the correct NTA, with the chosen workaround documented or removed.

Written by the indexing model from the issue text.

Assessment

Tech stack
sql
Domain
data-engineering, databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.