NYCPlanning / NYCPlanning/data-engineering

housing data for CPP

Open
#2,578 3 comments 0 reactions 1 assignee View on GitHub

@damonmcc is already working on this.

Since Aug 20, 2026.

db-knownprojects
Dominant language
Python
Stars
43
Forks
3
Avg merge
23h 3m
Merged PRs (30d)
44

Description

AE will add housing growth data to the Capital Projects Portal (CPP)

we're planning to generate two new csvs using Housing DB and KPDB (NTA and CD aggregations) with the following columns. the columns they originally requested are in bold:

  • geography_id
  • units_2020
  • completed_units_2015_2025
  • units_2025
  • projected_completed_units_2025_2035
  • projected_units_2035

notes

Figma - CPP - Housing Growth
AE parent issue: ae-cp-map - #547 Housing Growth Data CPP Integration

AE currently gets DE files from the edm-publishing bucket in folders organized by data product. it'd be nice to point them to one folder with one file per geography

if these new files were outputs of Housing DB and KPDB, they'd have to merge tables from the db-developments and a doesn't-exist-yet db-kpdb folders. maybe these could both be new outputs in KPDB since Housing DB is already an input to it

adding a new data product in products/ just for this seems like overkill, so maybe this is an excuse to start a more general "city model" data product?

KPDB builds are pushed to edm-private now. confirmed with Housing team that KPDB outputs are no longer considered sensitive, so pushing to edm-publishing seems fine

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.