PolicyEngine / PolicyEngine/chronicle
Fact provenance needs an asserting principal and root-artifact lineage so the microdata-derived-fact prohibition is enforceable
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 0
- Forks
- 1
- Avg merge
- 11h 48m
- Merged PRs (30d)
- 45
Description
Follow-up from the raw-microdata identity ADR (#222, gate round 3): the prohibition "no fact derived from raw microdata by Chronicle or a consumer enters Chronicle" is stated but not enforceable, because fact provenance today is free-form source metadata (source_name, source_table, source_file, extraction_method, …) with no asserting principal, no reference to the registered artifact the value came from, and no transitive root. validate_fact therefore cannot tell a publisher-published aggregate from a consumer aggregate that was derived from microdata through one or more intermediate artifacts.
Proposal
- Add to fact provenance (not to fact identity — identity payloads stay byte-identical):
asserted_by: the publisher id that asserted the value (the same id the source registry uses).root_artifacts: the registration tuples{source_id, package_id, year, sha256, filename}of every artifact the value ultimately comes from, transitively through derived artifacts.
validate_factrejects a fact whoseroot_artifactsinclude akind: microdata_releaseregistration unlessasserted_byis that release's publisher (the publisher-published-aggregate exception from the ADR), and rejects a fact with noasserted_byonce the field is required.- Source packages populate both fields from the manifest (publisher, registration tuple of the parsed artifact); derived artifacts carry their inputs' roots forward.
- Tests: a consumer-computed aggregate rooted in an FRS registration is refused; a DWP-published FRS table is accepted; a chain through an intermediate artifact is refused.
Depends on #227 (registration identity) and the epoch work (#228) for the provenance surface. The ADR (#222) references this issue as the enforcement mechanism for its decision point 3.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reading ADR #222 and dependencies #227 and #228, then inspect validate_fact, the fact-provenance model, and source-package manifest handling. Trace how derived artifacts currently record inputs. Done means the three proposed provenance tests pass: consumer-derived FRS aggregates are rejected, publisher-published FRS tables are accepted, and intermediate-artifact chains remain rejected.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- backend-api-design, data-engineering
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 42/100