PolicyEngine / PolicyEngine/chronicle

Fact provenance needs an asserting principal and root-artifact lineage so the microdata-derived-fact prohibition is enforceable

Open
#238 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement
Dominant language
Python
Stars
0
Forks
1
Avg merge
11h 48m
Merged PRs (30d)
45

Description

Follow-up from the raw-microdata identity ADR (#222, gate round 3): the prohibition "no fact derived from raw microdata by Chronicle or a consumer enters Chronicle" is stated but not enforceable, because fact provenance today is free-form source metadata (source_name, source_table, source_file, extraction_method, …) with no asserting principal, no reference to the registered artifact the value came from, and no transitive root. validate_fact therefore cannot tell a publisher-published aggregate from a consumer aggregate that was derived from microdata through one or more intermediate artifacts.

Proposal

  1. Add to fact provenance (not to fact identity — identity payloads stay byte-identical):
    • asserted_by: the publisher id that asserted the value (the same id the source registry uses).
    • root_artifacts: the registration tuples {source_id, package_id, year, sha256, filename} of every artifact the value ultimately comes from, transitively through derived artifacts.
  2. validate_fact rejects a fact whose root_artifacts include a kind: microdata_release registration unless asserted_by is that release's publisher (the publisher-published-aggregate exception from the ADR), and rejects a fact with no asserted_by once the field is required.
  3. Source packages populate both fields from the manifest (publisher, registration tuple of the parsed artifact); derived artifacts carry their inputs' roots forward.
  4. Tests: a consumer-computed aggregate rooted in an FRS registration is refused; a DWP-published FRS table is accepted; a chain through an intermediate artifact is refused.

Depends on #227 (registration identity) and the epoch work (#228) for the provenance surface. The ADR (#222) references this issue as the enforcement mechanism for its decision point 3.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reading ADR #222 and dependencies #227 and #228, then inspect validate_fact, the fact-provenance model, and source-package manifest handling. Trace how derived artifacts currently record inputs. Done means the three proposed provenance tests pass: consumer-derived FRS aggregates are rejected, publisher-published FRS tables are accepted, and intermediate-artifact chains remain rejected.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend-api-design, data-engineering
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.