PolicyEngine / PolicyEngine/chronicle

National generalization of the ACS SLD packages (all states, ~6.8k districts)

Open
#138 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
0
Forks
1
Avg merge
11h 48m
Merged PRs (30d)
45

Description

Follow-up to the Utah-pilot SLD packages (branch census-acs-sld-2024, populace#625). Verified mechanics for going national, so the generalization is mechanical once the pilot shape passes judge review:

  • Fetch: the API rejects in=state:* for SLD hierarchies but accepts comma-listed FIPS (in=state:01,02,...) — one canonical national witness per (table, chamber) works, same single-file shape as the CD packages. 50 states + DC; PR excluded per the populace state spine.
  • Scale (measured from the pilot's compact record-set shape — rows = districts with geography overrides, measures = columns with constraints): ~1,950 SLDU + ~4,833 SLDL districts → roughly 2.3MB YAML for the biggest package (S0101 lower) and ~3-4MB for the biggest witness; totals ~12MB YAML + ~12MB witnessed JSON across the six packages. CD-package-proportionate, no harness change needed.
  • Suppression handling to add: nationally, B19013 medians (and possibly sparse B19001 cells) will carry ACS suppression markers for thin districts. The pilot generator hard-fails on suppression; the national generator needs the skip-with-record path — suppressed cells excluded from the package with an explicit exclusions list in the package (never silently absent).
  • Special cases already structural in the pilot: NE unicameral and DC wards (no lower chamber), multi-member districts as Census tabulates them; NH floterial representation should be verified against the witnessed rows when the national fetch lands.
  • Ordering note: witness row order is the API's (sorted by state then district in practice); the generator asserts sortedness per state and derives row numbers from the parsed stream, so ordering drift is a loud failure.

Blocked behind: the pilot PR's judge review (shape adjudication) and the pending publish-raw R2 run for the pilot manifests.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reviewing the Utah-pilot SLD packages on branch census-acs-sld-2024 and the pending judge review and publish-raw R2 results. Then trace the pilot generator's suppression handling and national witness fetch, including NE, DC, and NH rows. Done means six national packages cover the 50 states and DC, record exclusions are explicit, and ordering and witnessed outputs pass without harness changes.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.