PolicyEngine / PolicyEngine/policyengine.py

Local reproducibility snapshots for researchers using the Python package

Open
#320 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
7
Forks
9
Avg merge
15h 51m
Merged PRs (30d)
9

Description

Context

At the 2026-04-21 meeting with Lars Vilhuber (transcript lines 65-75), the topic of local-copy and caching guidance for researchers using the Python package came up as a piece of reproducibility hygiene distinct from TRACE. Lars's framing: researchers running a simulation on their laptop should be able to preserve exactly what they ran (model + data + reform + environment), so that if they come back to it six months later, they can still reproduce their own work.

This is the non-TRACE version-identification workstream Casper spoke about separately (transcript 415-417): TRACE is for citations a reader cannot rerun; local version-identification is for the researcher themselves.

Policyengine-app#2832 implements the webapp-side version badge. This issue is the Python-package-side equivalent: help a researcher running policyengine locally keep a reproducible record of each run.

What to build

  1. A policyengine CLI command or helper that snapshots everything needed to reproduce a specific local run to a single directory:

    • Pinned package versions (pip freeze subset for pe.py + country + country-data)
    • The reform JSON (if any)
    • The h5 content hash (already in the release manifest)
    • The simulation output (results + optional per-household frame)
    • A short README documenting how to reproduce with the exact install line
  2. Documentation in household-api-docs showing researchers how to use this — distinct from the TRACE emission flow. The distinction matters because TRACE targets citation durability; local snapshots target "can I get back to my own work?"

  3. Default-on behavior for anyone using policyengine.calculate_household or policyengine.simulate via the Python API. A subdirectory under the working directory should be created automatically unless the user opts out. The cost of an extra megabyte of disk is worth the reproducibility gain.

Non-goals

  • Not TRACE. Local snapshots are not signed, not institutionally attested, not meant to serve as paper citations. They are researcher-local cache.
  • Not preservation-grade storage. Researchers responsible for their own backups.

Related

  • Transcript lines 65-75 and 415-417 at /tmp/aea-review/transcript.txt
  • PolicyEngine/household-api-docs#7 (Python / API docs — this issue adds a docs section)
  • PolicyEngine/policyengine-app#2832 (version badge, the webapp-side equivalent)
  • docs/trace-case-study.md (PR #315 — discusses this as adjacent to but not replaced by TRACE)

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the calculate_household and simulate Python API entry points, then read docs/trace-case-study.md and the related household-api-docs#7 discussion. Define the snapshot contents, opt-out behavior, and exact-install reproduction flow before choosing the CLI or helper surface. Done means local runs can produce a complete snapshot and the researcher documentation explains how to restore and reproduce it.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend-api-design, documentation
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.