frostney / frostney/GocciaScript

Nursery rooting: keep newly allocated values reachable until the next engine safe point

Open
#1,143 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

engine internal
Dominant language
Pascal
Stars
20
Forks
3
Avg merge
3d 4h
Merged PRs (30d)
45

Description

Summary

Replace the per-site TGocciaTempRoot discipline with an engine-level guarantee: every newly allocated GC-managed value stays reachable from an implicit nursery root until execution returns to a well-defined safe point (interpreter statement boundary, VM memory-pressure check, or builtin return). Native code would no longer need to root half-built results by hand.

Why

The collector is precise and marks only explicit roots — never the native stack — while any string allocation can trigger CollectForMemoryPressure mid-builtin (TryReserveExternalBytes in source/units/Goccia.GarbageCollector.pas). Every builtin that fills a freshly created container across such a safe point must therefore remember the temp-root idiom, and the track record shows this discipline does not hold at scale:

  • #1140 fixed ~35 use-after-free sites of exactly this shape (Object.keys bus error under --max-memory was the entry point), plus 7 more found in its review round (commit a84fe093).
  • A post-merge audit found a third wave the sweep still missed: the JSONL chunk builder, the JSON5 reviver/replacer paths, and the Promise.allSettled entry objects (follow-up in progress), with the recursive JSON/JSON5/YAML/TOML parsers still needing an interprocedural check.
  • #563 (closed) was the same defect in the VM's saved-closure locals — the pattern predates the builtin sweep.
  • The codebase now carries ~587 manual rooting calls across 84 files, and every new builtin must get this right by hand. The failure mode is silent heap corruption that only reproduces near a memory ceiling, which is the worst kind of bug to field-diagnose.

Current behavior

Correctness depends on each native call site pairing InitializeTempRoot / AddTempRootIfNeeded / RemoveTempRootIfNeeded (or raw AddTempRoot/RemoveTempRoot) around every allocation window. Missed sites crash as Fatal error: Bus error or misaligned data access (production) or Access violation / Object reference is Nil (checked builds) when a collection lands inside the window.

Expected behavior

A value allocated during native execution cannot be collected before the engine passes a safe point at which it is either stored into a traced structure or provably dead. Hand-written temp roots become unnecessary for the common build-a-result-and-return shape (explicit roots remain for values intentionally held across JS re-entry, e.g. iterator protocol state).

Scope notes

Design sketch (to be validated, not prescribed):

  • The GC already tracks registration order (FManagedObjects + watermark, used by CollectYoung). A nursery could be "every object registered since the last safe-point flush is implicitly rooted": mark from Watermark-style index to end during MarkRoots, and advance the flush index at interpreter statement boundaries, at the VM's existing MEMORY_PRESSURE_CHECK_INTERVAL check (Goccia.VM.pas ~17480), and on builtin return (TGocciaNativeFunctionValue.Call already brackets every native call).
  • Key risk: collections triggered inside a long native loop (the very case #1140 fixed) must still be able to reclaim garbage created earlier in that same loop, or CSV-parsing a huge file under a ceiling would refuse where it previously succeeded. Options include flushing the nursery at charge points where the requesting value is the only protected object, or a bounded nursery with an explicit spill-to-refusal policy. This tension is the core design question.
  • Interaction with #1137's budget gate: RequireNativeBytes is check-and-raise and never collects, so it needs no changes, but nursery retention slightly raises peak BytesAllocated between flushes; the pressure-collection reserve (MEMORY_PRESSURE_COLLECTION_MIN_RESERVE) may need tuning.
  • Migration: land the mechanism, then remove per-site temp roots incrementally behind the existing test coverage — scripts/test-cli.ts already sweeps own-key enumeration and the builder paths across memory limits with a negative-controlled crash detector, which is the regression harness for exactly this class.
  • Non-goals: no change to JS-visible semantics; uncatchable budget refusals from #1137 stay as they are.

Related: #1140, #1137, #563.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with source/units/Goccia.GarbageCollector.pas, Goccia.VM.pas around the MEMORY_PRESSURE_CHECK_INTERVAL check, and TGocciaNativeFunctionValue.Call; then run scripts/test-cli.ts to review the existing memory-limit regression coverage. Done means a validated safe-point nursery design that preserves collection behavior during long native loops, integrates with the budget gate, and passes the existing crash-detector tests.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript
Domain
compilers
Issue type
Refactor
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.