PolicyEngine / PolicyEngine/chronicle

build-bundle emits source_label concept alignments without authority; build-consumer-artifact refuses them under consumer_fact.v3

Open
#277 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
0
Forks
1
Avg merge
11h 48m
Merged PRs (30d)
45

Description

What happens

At c5e5bf8 (main as of 18 September 2026), chronicle build-bundle emits consumer facts whose concept_alignment object has relation: source_label and no authority, and reports the bundle valid: true. chronicle build-consumer-artifact then refuses the same rows, because policyengine_chronicle/schemas/consumer_fact.v3.schema.json lists authority under concept_alignment.required (it has since cafc583, the commit that introduced consumer_fact.v3):

ValueError: Consumer fact row 228 of .../consumer_facts.jsonl failed schema validation at 'concept_alignment': 'authority' is a required property

So the two commands disagree about what a valid v3 row is, and a US consumer feed that build-bundle produces cannot become a consumer artifact.

Where

Measured on the US feed Microcosm now pins (PolicyEngine/microcosm#955: 586 record-set/period pairs, 39,158 rows, built from 62 targeted build-bundle --year Y --source <package> runs at c5e5bf8). 994 rows fail, all with relation: source_label, in three packages:

package_id rows without authority
cms-medicaid-chip-monthly-enrollment-dataset 515
census-b01001-female-age-2023 468
jct-tax-expenditures-2024 11

Every other row in that feed either carries authority or has no concept_alignment at all (which the schema permits).

Effect downstream

Microcosm's --base-h5 release arm accepts a bare consumer_facts.jsonl, so the re-pin in PolicyEngine/microcosm#955 proceeds as a bare feed (manifest_sha256: null). Its --exact-k arm requires an artifact directory with manifest.json (--ledger-manifest-sha256), so that arm cannot use any US feed until this is resolved.

Options

  1. Record an authority for the source_label alignments in those three packages, so the rows satisfy the schema as written. Microcosm then re-pins at that commit with a manifest.
  2. Relax the schema so authority is optional when relation is source_label, if a source's own label genuinely has no external authority to cite.

Either way, build-bundle and build-consumer-artifact should agree: a bundle reported valid should build an artifact.

Acceptance

chronicle build-consumer-artifact --facts <rebuilt US feed> succeeds on a feed built from us/chronicle_feed_scope.json in PolicyEngine/microcosm at the fixing commit; Microcosm's tools/build_us_chronicle_feed.py then runs without --skip-artifact and the pin records manifest_sha256.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with policyengine_chronicle/schemas/consumer_fact.v3.schema.json and trace validation in build-bundle and build-consumer-artifact. Rebuild the US feed from us/chronicle_feed_scope.json, then run build-consumer-artifact --facts on it and tools/build_us_chronicle_feed.py without --skip-artifact. Done means the commands agree on validity and the pin records manifest_sha256.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
build-system, data
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.