frostney / frostney/GocciaScript
Nursery rooting: keep newly allocated values reachable until the next engine safe point
Nobody has claimed this yet.
- Dominant language
- Pascal
- Stars
- 20
- Forks
- 3
- Avg merge
- 3d 4h
- Merged PRs (30d)
- 45
Description
Summary
Replace the per-site TGocciaTempRoot discipline with an engine-level guarantee: every newly allocated GC-managed value stays reachable from an implicit nursery root until execution returns to a well-defined safe point (interpreter statement boundary, VM memory-pressure check, or builtin return). Native code would no longer need to root half-built results by hand.
Why
The collector is precise and marks only explicit roots — never the native stack — while any string allocation can trigger CollectForMemoryPressure mid-builtin (TryReserveExternalBytes in source/units/Goccia.GarbageCollector.pas). Every builtin that fills a freshly created container across such a safe point must therefore remember the temp-root idiom, and the track record shows this discipline does not hold at scale:
- #1140 fixed ~35 use-after-free sites of exactly this shape (
Object.keysbus error under--max-memorywas the entry point), plus 7 more found in its review round (commita84fe093). - A post-merge audit found a third wave the sweep still missed: the JSONL chunk builder, the JSON5 reviver/replacer paths, and the
Promise.allSettledentry objects (follow-up in progress), with the recursive JSON/JSON5/YAML/TOML parsers still needing an interprocedural check. - #563 (closed) was the same defect in the VM's saved-closure locals — the pattern predates the builtin sweep.
- The codebase now carries ~587 manual rooting calls across 84 files, and every new builtin must get this right by hand. The failure mode is silent heap corruption that only reproduces near a memory ceiling, which is the worst kind of bug to field-diagnose.
Current behavior
Correctness depends on each native call site pairing InitializeTempRoot / AddTempRootIfNeeded / RemoveTempRootIfNeeded (or raw AddTempRoot/RemoveTempRoot) around every allocation window. Missed sites crash as Fatal error: Bus error or misaligned data access (production) or Access violation / Object reference is Nil (checked builds) when a collection lands inside the window.
Expected behavior
A value allocated during native execution cannot be collected before the engine passes a safe point at which it is either stored into a traced structure or provably dead. Hand-written temp roots become unnecessary for the common build-a-result-and-return shape (explicit roots remain for values intentionally held across JS re-entry, e.g. iterator protocol state).
Scope notes
Design sketch (to be validated, not prescribed):
- The GC already tracks registration order (
FManagedObjects+ watermark, used byCollectYoung). A nursery could be "every object registered since the last safe-point flush is implicitly rooted": mark fromWatermark-style index to end duringMarkRoots, and advance the flush index at interpreter statement boundaries, at the VM's existingMEMORY_PRESSURE_CHECK_INTERVALcheck (Goccia.VM.pas~17480), and on builtin return (TGocciaNativeFunctionValue.Callalready brackets every native call). - Key risk: collections triggered inside a long native loop (the very case #1140 fixed) must still be able to reclaim garbage created earlier in that same loop, or CSV-parsing a huge file under a ceiling would refuse where it previously succeeded. Options include flushing the nursery at charge points where the requesting value is the only protected object, or a bounded nursery with an explicit spill-to-refusal policy. This tension is the core design question.
- Interaction with #1137's budget gate:
RequireNativeBytesis check-and-raise and never collects, so it needs no changes, but nursery retention slightly raises peakBytesAllocatedbetween flushes; the pressure-collection reserve (MEMORY_PRESSURE_COLLECTION_MIN_RESERVE) may need tuning. - Migration: land the mechanism, then remove per-site temp roots incrementally behind the existing test coverage —
scripts/test-cli.tsalready sweeps own-key enumeration and the builder paths across memory limits with a negative-controlled crash detector, which is the regression harness for exactly this class. - Non-goals: no change to JS-visible semantics; uncatchable budget refusals from #1137 stay as they are.
Related: #1140, #1137, #563.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with source/units/Goccia.GarbageCollector.pas, Goccia.VM.pas around the MEMORY_PRESSURE_CHECK_INTERVAL check, and TGocciaNativeFunctionValue.Call; then run scripts/test-cli.ts to review the existing memory-limit regression coverage. Done means a validated safe-point nursery design that preserves collection behavior during long native loops, integrates with the budget gate, and passes the existing crash-detector tests.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- javascript
- Domain
- compilers
- Issue type
- Refactor
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100