frostney / frostney/GocciaScript
Add define-gated fault injection so the runner's integrity abort has CI coverage
Nobody has claimed this yet.
- Dominant language
- Pascal
- Stars
- 20
- Forks
- 3
- Avg merge
- 3d 4h
- Merged PRs (30d)
- 45
Description
Summary
Give the test runner's integrity-fault abort automated coverage by compiling a fault-injection hook only under a dedicated build define (e.g. -dINTEGRITY_FAULT_INJECTION), and add a CI job that builds that variant and runs the abort contract matrix.
Why
Since the stack #1159 hardening (ADR 0109 host tier), the runner aborts the whole run — distinct two-line stderr diagnostic, no summary, no JSON envelope, exit 70 — when an engine-integrity fault escapes a test file. That behavior currently has zero automated coverage: it was verified by a temporary, hand-applied injection patch (recorded in ADR 0109's seam finding), so a regression that silently drops the abort would not be caught by CI. The original decision not to ship an injection hook was about the production binary; a define-gated hook that shipped targets never set preserves that property, following the same separation the repo already uses for special-purpose targets (fuzz harness, sandbox runner).
Current behavior
AbortRunOnIntegrityFault/ the worker-arm terminate path insource/app/GocciaTestRunner.dprare unreachable by any committed test.- The verified contract lives only in ADR 0109's recorded manual matrix (all mode×jobs shapes, multi-worker faults, abandoned-worker bypass — all exit 70, exactly two diagnostic lines, empty stdout).
scripts/test-cli.tslocks the negative contract only (a failing file still exits 1 with an unchanged report and noIntegrity fault:text).
Expected behavior
- A build define compiles a fault-injection entry point into the runner (its only purpose: raise an
EObjectCheck-class fault inside a chosen test file's execution). The define is set by no shipped build target; the production binary is byte-unaffected. - A CI job builds the injection variant and asserts the abort contract: exit 70, exactly two diagnostic lines naming the faulting file, zero summary/envelope output, across
--jobs=1/--jobs=N, both--modes,--output=json, and a multi-fault shape (single diagnostic via the first-fault gate). - While in the file: the two refusal-family tests in
source/units/Goccia.MemoryLimit.Test.passtill carrying the deadcaughtvariable adopt the witness assertion the integrity tests use (one line each; the witness native is already injected).
Scope notes
- In scope: the deterministic abort contract. Out of scope: the stray-line/truncation race regressions fixed on stack #1159 — the diagnosis proved that window is a Heisenbug (an observation probe made it stop reproducing), so its evidence remains the recorded loop transcript, not a CI assertion.
- Constraint: the hook must be provably absent from non-injection builds (e.g. a
strings/symbol check in the CI job, or the define simply not existing in any build.pas target). - Related: ADR 0109 (host tier + seam finding), stack #1159 (#1163, #1164, #1165),
docs/testing.md"Engine-integrity faults abort the run".
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with source/app/GocciaTestRunner.dpr and the integrity-abort contract described in ADR 0109 and docs/testing.md. Review scripts/test-cli.ts and source/units/Goccia.MemoryLimit.Test.pas, then inspect the existing CI targets for special-purpose builds. Done means the define-gated variant exercises the stated mode, jobs, output, and multi-fault matrix while non-injection builds exclude the hook and the dead caught variables are replaced with witness assertions.
Written by the indexing model from the issue text.
Assessment
- Domain
- ci-cd, devtools, testing-qa
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Clearly specified
- Newbie friendliness
- 35/100