TaxBenefitSystem construction permanently leaks ~5,600 sys.modules entries (~55-60MB) per build — unreclaimable by gc

Every TaxBenefitSystem construction permanently leaks ~5,600 sys.modules entries (~55-60 MB RSS, immune to gc)

TaxBenefitSystem.add_variables_from_file registers every variable module under a name unique to the system instance and never evicts it:

# taxbenefitsystems/tax_benefit_system.py (3.26.11: L288-294)
module_name = f"{id(self)}_{hash(os.path.abspath(file_path))}_{file_name}"
spec = importlib.util.spec_from_file_location(module_name, file_path)
module = importlib.util.module_from_spec(spec)
sys.modules[module_name] = module
spec.loader.exec_module(module)

sys.modules is process-immortal, so each constructed system strands one full set of variable-module objects — with all their module-level globals — beyond the reach of any garbage collection, even after the system itself is deleted and fully collected.

Measurement (policyengine-us 1.764.6, policyengine-core 3.26.11, py3.14)

Three sequential CountryTaxBenefitSystem() builds in one process, del system + full gc.collect() after each:

+5,603 permanent entries and ~55-60 MB permanent RSS per construction. The full collect frees the system's own cyclic graph (~3.02 M objects) every time — the sys.modules set is what survives. A concrete exemplar retained via referrer chains: one 32,355-row HUD FMR lookup DataFrame per build, pinned by <id>_<hash>_hud_fair_market_rent.FMR_LOOKUP.

Why it matters

Any workload that constructs systems repeatedly — reform simulations (Simulation.__init__ builds a fresh system whenever reform is not None), batched pipelines, long-lived services — accumulates this floor linearly and invisibly. It was the dominant term in PolicyEngine/populace#456: a dense calibration build constructing ~1,500 reform-batch systems leaked ~150 GB of unreclaimable RSS and OOM'd 128 GB and 256 GB machines (fixed on the populace side by reusing one system per reform family, but the engine-side leak remains for every other consumer).

Possible fixes

1. Evict after exec: sys.modules.pop(module_name, None) in a finally: once exec_module returns. The Variable classes keep everything they need alive through their methods' __globals__; the registry entry adds nothing but immortality. (If eviction races something that re-imports by name, keep the entry only for the duration of the directory load.)
2. Cache variable modules by file path (not per-system id(self)): systems constructed from the same country package would share one exec per file — this also removes ~all of the ~4 s per-construction variable-loading cost for the second and later systems, at the cost of sharing module-global state between systems (needs an audit of country packages for mutable module globals).

Option 1 is the safe minimal fix; option 2 is the performance fix if module globals are confirmed immutable-by-convention.

Profile scripts and the full populace-side incident analysis: PolicyEngine/populace#456 (comment with the measurement tables).

Start in taxbenefitsystems/tax_benefit_system.py, especially add_variables_from_file around lines 288-294, and review the module-loading lifecycle and the two proposed approaches. Reproduce repeated TaxBenefitSystem construction with garbage collection and inspect sys.modules growth. Done means variable loading no longer leaves one permanent module set per system while existing variable behavior remains intact.