PolicyEngine / PolicyEngine/policyengine-api

Track Stage 12 automatic result comparisons

Open
#3,841 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
18
Forks
33
Avg merge
23h 40m
Merged PRs (30d)
22

Description

Objective

Extend the temporary Stage 12 comparison records so the simulation service can record whether its aggregate result exactly matches the corresponding production result, without making Stage 12 data authoritative or serving it to users.

Scope

  • Use comparison-run terminology in application modules while retaining the already-deployed stage12_evaluation_* table names, enum names, and evaluation_id columns until Stage 14.
  • Add comparison status, artifact URI and digest, schema version, completion time, and bounded error fields to the temporary parent record.
  • Keep lifecycle updates and result-comparison updates separate so a comparison failure cannot change a successful Stage 12 calculation result.
  • Add an autogenerated additive v2 Alembic revision and cover upgrade, downgrade, re-upgrade, existing-row defaults, PostgreSQL behavior, and schema drift.
  • Document automatic execution, exact result comparison, STAGE12_ENABLED, temporary records, and the absence of periodic cleanup.

Acceptance criteria

  • Existing public API behavior and existing v1 calculation authority do not change.
  • Normal API services never read the temporary comparison tables.
  • Production simulations, reports, report_runs, and association records do not depend on the temporary tables.
  • The simulation runtime can update comparison metadata through the existing restricted PostgreSQL role without schema-migration authority.
  • The temporary rows remain until the Stage 14 follow-up change removes the tables; private objects remain subject to the existing 30-day storage lifecycle.
  • Focused tests, PostgreSQL lifecycle tests, migration checks, and generated migration-contract checks pass.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by tracing the existing Stage 12 temporary records, the simulation runtime, and the current Alembic migration contract. Review the restricted PostgreSQL role and focused migration and lifecycle tests before changing scope. Done means additive v2 migration coverage, separated lifecycle and comparison updates, PostgreSQL checks, schema-drift checks, and documentation of the temporary behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
postgresql, python
Domain
backend-api-design, databases, testing
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.