PolicyEngine / PolicyEngine/policyengine-core
Fix parameter validation when using variable references in breakdown metadata
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 22
- Forks
- 30
- Avg merge
- 14h 33m
- Merged PRs (30d)
- 7
Description
Issue with parameter validation when using variable references in breakdown parameters
Background
When implementing scale parameters with integer output values in PolicyEngine US, I encountered an error during parameter validation. This occurs when trying to use variable references (like variable enumerations) in the 'breakdown' section of parameter metadata.
Error details
The error occurs in the homogenize_parameter_node function, with the following traceback:
UnboundLocalError: cannot access local variable 'possible_values'
The specific issue appears when parameter files have a breakdown structure where:
- The parameter references an enumerated variable (like age groups)
- The parameter is meant to return integer values
Problematic patterns
We've identified several parameter structures that trigger this validation error:
- Using variable references in breakdown metadata:
childcare_market_rates:
description: Maximum market rates for child care by age group
values:
2022-01-01:
# Error when using variable reference
breakdown:
nc_scca_age_group: # Reference to a variable enum
1: 500 # Values per age group
2: 650
3: 700
4: 580
- Using boolean keys as parameter names:
fpg_limit:
description: FPG percentage limits
values:
2022-01-01:
# Error when using boolean keys directly
True: 200
False: 100
Solutions and workarounds
We've discovered several effective workarounds:
- Use
range()instead of variable references in breakdowns:
childcare_market_rates:
description: Maximum market rates for child care by age group
values:
2022-01-01:
# Works properly
breakdown:
range(1, 5): # Python expression instead of variable reference
1: 500
2: 650
3: 700
4: 580
- Split complex parameters into separate, simpler parameter files:
# fpg_limit_preschool.yaml
fpg_limit_preschool:
description: FPG percentage limit for preschool children
values:
2022-01-01: 200
# fpg_limit_school_age.yaml
fpg_limit_school_age:
description: FPG percentage limit for school age children
values:
2022-01-01: 100
- Use string names instead of boolean keys:
fpg_limit:
description: FPG percentage limits
values:
2022-01-01:
breakdown:
is_eligible: # String names work better
'eligible': 200
'not_eligible': 100
- Use
/1instead ofintinrate_unitfor scale parameters:
amount:
description: Benefit amount in dollars
metadata:
unit: currency-USD
period: year
rate_unit: /1 # Works better than rate_unit: int
values:
2022-01-01: 250
Potential fix
The core issue appears to be in the parameter validation function that doesn't properly handle variable references in breakdown metadata. Possible fixes:
- Add better error handling when processing breakdowns with variable references
- Support dynamically resolving variable references during validation
- Update documentation to clearly explain supported parameter structures
- Add tests for various parameter structure patterns
Real-world impact
This issue has been encountered implementing state programs (like NC's Subsidized Child Care Assistance) where parameters need to be defined by enumerated categories. The workarounds reduce readability and maintainability.
Related PRs
- PolicyEngine/policyengine-us#5575
- PolicyEngine/policyengine-us#5641
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the homogenize_parameter_node function and reproduce the UnboundLocalError using a parameter breakdown that references an enumerated variable, as described in the issue. Compare this with the working range() form and the boolean-key example; done means validation handles these structures without the error and tests cover the reported patterns.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- backend
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100