PolicyEngine / PolicyEngine/cliff-watch
Intermittent 'rules-engine formulas must be deterministic' errors from /api/series
Nobody has claimed this yet.
- Dominant language
- JavaScript
- Stars
- 0
- Forks
- 0
- PR merge metrics
- No merged PRs in 30d
Description
Twice on 2026-07-03 (Europe/Brussels morning), in-app series calculations against the deployed API failed with policyengine-core's randomness guard, e.g.:
Calculation failed: The formula for 'slcsp_age_0' called numpy.random.seed(), but rules-engine formulas must be deterministic functions of their inputs. Remove the random call. If you need a stochastic input, compute it once when building the dataset (with a seeded generator) and store it as an input variable instead.
and the same wording naming age_head during a cliff-refinement request. Both appeared in the frontend as "Calculation failed: …" (the api/* error prefix), for an NC household (adult 30, children 4 and 8, ZIP 27601, all programs) swept 0–100k.
The guard landed in policyengine-core on 2026-06-04 (randomness_guard.py, "Forbid randomness inside variable formulas"); this repo floats policyengine-us>=1.715.2, so deployments resolve whatever us/core pair exists at build time.
What I could not do is reproduce it on demand: direct POSTs to /api/series on the current production deployment succeed for the same household — full 0–100k sweep at step 1000, refinement-shaped requests with min_earned_income, and ACA-region ranges where slcsp_age_0 computes (aca_ptc > 0). That leaves an intermittent trigger — plausibly a conditional randomness path (e.g., a take-up or imputation branch) that only some request shapes or instance states reach.
Suggestions:
- Pin
policyengine-us(and transitively policyengine-core) to an exact known-good pair inrequirements.txt/pyproject.tomlso deploys stop drifting, and bump deliberately. - Grep the pinned policyengine-us for
np.randomreachable from formulas (slcsp_age_0,age_headdependency chains) and either fix upstream or pin below the offending release. - If it recurs, capture the full traceback server-side — the guard's error names the innermost formula but not the caller that seeded.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start at the /api/series entry point and inspect the resolved policyengine-us/policyengine-core versions in requirements.txt or pyproject.toml. Read randomness_guard.py, then grep the pinned policyengine-us dependency chains for np.random near slcsp_age_0 and age_head; if the issue recurs, capture the server-side traceback. Done means the offending dependency pair or upstream formula is identified and the deployment no longer reaches nondeterministic formulas.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- numpy, python
- Domain
- api, backend
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 45/100