PolicyEngine / PolicyEngine/microcosm

Replace all remaining policyengine-us-data artifacts with Populace

Open
#204 17 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
0
Forks
4
Avg merge
1d 3h
Merged PRs (30d)
94

Description

Goal

Make Populace the complete replacement for policyengine-us-data, including the national dataset, local/state/congressional-district outputs, release manifests, diagnostics, and the .py default stack contract. Once those are live, archive the old data repo rather than continuing old-stack fixes.

Scope

  • Publish a Populace US release that passes release gates on the small national support frame while carrying the full current fiscal target surface, with congressional-district rows diagnostic-only until the support frame is designed to satisfy them.
  • Add standard common-surface scoring so every candidate is compared against the incumbent on the same target registry before publication.
  • Replace inherited policyengine-us-data artifacts in .py manifests with Populace-owned artifacts or explicit temporary inheritance notes.
  • Provide Populace-owned state, local, and congressional-district dataset outputs or a clear loader strategy that supersedes policyengine-us-data/states/*, districts/*, and related local files.
  • Preserve useful release-manifest and compatibility metadata, but keep incumbent/eCPS benchmark comparisons outside the live Populace repo.
  • Close or migrate remaining old-repo issues/PRs only as Populace specs, source loaders, target profiles, diagnostics, or release-contract work. Do not port old implementation structure.

Current blockers / active work

  • A guarded 32k+ target-surface build is queued from current main after #203; it will start when memory pressure from the active L0 benchmark clears.
  • The full CD target surface should not block the small national release solely on CD zero-support rows, but those rows must remain visible in diagnostics.
  • We still need an explicit artifact-level replacement table for every policyengine-us-data artifact currently referenced by .py or historical release manifests.

Acceptance criteria

  • The current .py default US dataset is Populace, with verified compatible policyengine-us/policyengine-core versions and release manifest metadata.
  • Populace has a documented replacement path for national, state, district, city/local, long-term, and auxiliary artifacts previously supplied by policyengine-us-data.
  • Release gates include source coverage, formula-owned export assertions, target materialization diagnostics, common-surface incumbent comparison, and critical-target tolerances.
  • All open policyengine-us-data issues/PRs are either linked to Populace issues, explicitly marked benchmark-only/out-of-scope, or closed as obsolete.
  • policyengine-us-data is archived or made read-only with a README pointing to Populace and any benchmark repo for historical comparisons.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reviewing the .py manifests, the current main state after #203, and the queued 32k+ target-surface build. Map every policyengine-us-data artifact to a Populace artifact or documented temporary inheritance note, then examine the release gates and diagnostics. Done means the default US dataset is Populace, replacement paths and compatibility metadata are documented, and the old repository is archived or read-only with a pointer to Populace.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
build-system, data-engineering, release
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.