PolicyEngine / PolicyEngine/policyengine-us
CI can miss failures against newer policyengine-core because uv.lock pins stale core
- Dominant language
- Python
- Stars
- 162
- Forks
- 212
- Avg merge
- 3d 3h
- Merged PRs (30d)
- 97
Description
## Summary
Household API staging deployment tests recently surfaced a `policyengine-core` compatibility failure that `policyengine-us` CI did not catch. The likely reason is that `policyengine-us` test jobs install from the committed `uv.lock`, which currently pins an older `policyengine-core`, while downstream consumers can resolve newer compatible `policyengine-core` versions from the package's lower-bound dependency range.
## Observed downstream failure
The household API staging tests failed with:
```text
The formula for 'is_ssi_recipient_for_medicaid' called numpy.random.seed(), but rules-engine formulas must be deterministic...
```
That failure appeared in the household API deployment path after resolving newer package versions, including a newer `policyengine-core` version than the one used by the locked `policyengine-us` test environment.
Related temporary downstream pin:
- PolicyEngine/policyengine-household-api#1575
- PolicyEngine/policyengine-household-api#1576
## Why this matters for policyengine-us
`policyengine-us` declares a compatible lower-bound dependency on `policyengine-core`, but normal CI runs appear to use `uv sync --extra dev`, which respects the committed lock file. In the current checkout, `uv.lock` pins `policyengine-core==3.26.0`, so PR/push tests can pass against that stale core even when downstream installs resolve a newer compatible `policyengine-core` that changes runtime behavior.
This means `policyengine-us` CI may not catch regressions introduced by newer `policyengine-core` versions until a downstream application, such as `policyengine-household-api`, installs and exercises that newer core.
## Suggested fixes
- Add a CI job that tests against the latest compatible `policyengine-core`, independent of the committed lock file.
- Consider a scheduled lock refresh/upgrade workflow that validates the package against upgraded dependencies.
- Add a focused regression test for the Medicaid/SSI path that triggered the household API failure, especially around `is_ssi_recipient_for_medicaid`.
- Consider porting or mirroring relevant household API partner/customer-input fixtures into `policyengine-us` when they exercise package behavior that normal YAML tests miss.
- If newer `policyengine-core` behavior is intentionally incompatible, add an upper bound or compatibility gate until `policyengine-us` is updated.
## Notes
The failing formula itself does not obviously contain a direct random seed call in the current source. This issue is about the test-environment gap first: `policyengine-us` should catch compatibility failures against newer allowed `policyengine-core` versions before they reach downstream deployments.
Contributor guide
Research direction
Start with uv.lock and the CI job that runs `uv sync --extra dev`; compare its pinned policyengine-core with the compatible versions allowed by policyengine-us. Review the reported `is_ssi_recipient_for_medicaid` failure and existing test coverage, then define completion as CI exercising a newer compatible core and catching the relevant compatibility regression before downstream deployment.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- ci-cd, testing
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100