PolicyEngine / PolicyEngine/policyengine-core

Proposal: deterministic PolicyEngine YAML test -> portable fixture converter

Open
#514 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
22
Forks
30
Avg merge
14h 33m
Merged PRs (30d)
7

Description

I have a small deterministic converter prototype that may be useful for PolicyEngine's existing validation and cross-model comparison work: it converts supported PolicyEngine YAML tests into an intermediate fixture JSON format and back to canonical PolicyEngine YAML data.

Current prototype supports
  • PolicyEngine YAML test → intermediate fixture JSON → PolicyEngine YAML test round-trips for a documented subset
  • Scalar and nested entity input/output maps
  • Absolute error margins
  • Optional crosswalk remapping for cross-engine conversion when every variable has an explicit mapping
  • Loud rejection for unsupported constructs (reforms, extensions, relative error margins, dotted input keys treated as inline parameter reforms, lists, nulls, expression strings)

I verified the current PolicyEngine YAML loader from policyengine-core commit f761573c2a13adecc3826be04af1980d13657e1d (policyengine_core/tools/test_runner.py) before defining the subset.

Corpus evidence
  • Source corpus: 10 YAML files from PolicyEngine/policyengine-us commit fc64cef64ab55c3c48309c7fb304c35e5f3c9184
  • Converted files: 10/10
  • Rejected files: 0/10 in the selected v0.1 subset corpus
  • Generated fixture JSON is validated with a JSON Schema validator in CI

This is intended as a migration/cross-validation utility, not a replacement for PolicyEngine's native test format. The converter refuses ambiguous or engine-specific constructs rather than silently dropping them.

Questions

Would PolicyEngine maintainers be open to this living as:

  1. an external utility with docs/examples that reference PolicyEngine's current YAML test shape, or
  2. a small upstream docs/testing contribution if useful?

Given PolicyEngine's existing validation culture, the highest-value next step may be a narrow demonstration on a package slice rather than a broad format discussion.

Evidence packet (external): https://github.com/edithatogo/rulesandprocesses/tree/main/converters/fixtures

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reading policyengine_core/tools/test_runner.py and the external converters/fixtures evidence packet to understand the current YAML test shape and proposed subset. Determine whether the work belongs upstream or remains external, and define acceptance around a narrow demonstration, supported round-trips, explicit rejection of unsupported constructs, and JSON Schema validation in CI.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
testing, tooling
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.