PolicyEngine / PolicyEngine/microcosm

Stand up the populace-be release channel

Open
#265 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
0
Forks
4
Avg merge
1d 3h
Merged PRs (30d)
94

Description

Part of #259 (populace-be epic).

What

Stand up the BE release/publishing channel, reusing the US/UK machinery:

  • Private HF dataset repo (policyengine/populace-be-private or per the licence issue's naming) holding populace_be_<year>.h5 + full release contract artifacts: build_manifest.json, release_manifest.json, calibration_diagnostics.json, source coverage (generalize the us_source_coverage.json naming to country-neutral), demographics.json, reform_validation.json.
  • latest.json uploaded last as the atomic pointer; HF revision tag per release.
  • populace-publish-release works unchanged for BE (contract validation raising all failures at once; --repo-id targeting the private repo only, enforced by the destination guard from the SILC issue).
  • Staging telemetry repo (populace-be-staging analog) so the dashboard staging monitor works.
  • Slack release alert.
  • DatasetSpec registry entry: ("be", <year>, "compact") → private repo, engine adapter class per the Axiom adapter issue, so populace.data.load("be", <year>) resolves for authorized tokens.
  • Naming: data_build_id convention; no ordinals.

Public/private boundary

Everything in releases/<build_id>/ except the H5 and calibration NPZ is aggregate-only and safe to expose to the dashboard; confirm each artifact against the licence posture before first publish (per-record diagnostics are excluded from BE releases).

Acceptance

  • End-to-end: build → contract validation → private publish → latest.json flip → Slack alert → dashboard reads it (with token).
  • A dry-run publish to a scratch private repo is exercised in CI or a documented manual runbook.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reading the existing US/UK release machinery, the populace-publish-release command, and the DatasetSpec registry, then trace the SILC destination guard and Axiom adapter dependencies. Verify how the staging monitor and Slack alert are wired. Done means the stated build-to-dashboard flow works for BE, including a scratch private-repo dry run or documented runbook.

Written by the indexing model from the issue text.

Assessment

Tech stack
huggingface, python
Domain
backend, cloud, data, release
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.