PolicyEngine / PolicyEngine/policyengine-core
Storage keys need a structured (branch, period) scheme: str(period) is lossy and separators collide with branch names
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 22
- Forks
- 30
- Avg merge
- 14h 33m
- Merged PRs (30d)
- 7
Description
#523 / #525 fix the crash where get_known_periods/get_known_branch_periods split storage keys on every colon, but the underlying key scheme has deeper, pre-existing structural problems (surfaced in cross-family review of #525):
str(period)is lossy for day-anchored periods. Keys embedf"{branch_name}:{period}", andPeriod.__str__drops the day for e.g.month:2027-11-15:2/year:2027-11-15:2, so distinct valid periods can collide in the cache and the readers reconstruct date-shifted periods.- Branch names have no enforced grammar.
Simulation.get_branchaccepts arbitrary names; a branch name containing:breakssplit(":", 1)parsing in the readers and indelete. OnDiskStoragehas the same defect class with_as the separator ({branch}_{period}+ unconstrainedsplit("_")): a branch name likemy_reformmis-parses, and complex-period colons produce awkward filenames.
The clean fix is structured keys — (branch_name, Period) tuples in memory and an escaped/encoded filename scheme on disk — rather than string round-trips. That is a storage-format refactor deserving its own design + review, hence this issue rather than a rider on #525.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by tracing get_known_periods and get_known_branch_periods, then inspect Simulation.get_branch, delete, OnDiskStorage, and Period. Design structured in-memory keys using (branch_name, Period) and an escaped or encoded on-disk filename scheme. Done means distinct day-anchored periods and branch names containing separators round-trip without collisions or parsing errors.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- backend
- Issue type
- Refactor
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100