frostney / frostney/GocciaScript

Add define-gated fault injection so the runner's integrity abort has CI coverage

Open
#1,166 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

automated internal
Dominant language
Pascal
Stars
20
Forks
3
Avg merge
3d 4h
Merged PRs (30d)
45

Description

Summary

Give the test runner's integrity-fault abort automated coverage by compiling a fault-injection hook only under a dedicated build define (e.g. -dINTEGRITY_FAULT_INJECTION), and add a CI job that builds that variant and runs the abort contract matrix.

Why

Since the stack #1159 hardening (ADR 0109 host tier), the runner aborts the whole run — distinct two-line stderr diagnostic, no summary, no JSON envelope, exit 70 — when an engine-integrity fault escapes a test file. That behavior currently has zero automated coverage: it was verified by a temporary, hand-applied injection patch (recorded in ADR 0109's seam finding), so a regression that silently drops the abort would not be caught by CI. The original decision not to ship an injection hook was about the production binary; a define-gated hook that shipped targets never set preserves that property, following the same separation the repo already uses for special-purpose targets (fuzz harness, sandbox runner).

Current behavior

  • AbortRunOnIntegrityFault / the worker-arm terminate path in source/app/GocciaTestRunner.dpr are unreachable by any committed test.
  • The verified contract lives only in ADR 0109's recorded manual matrix (all mode×jobs shapes, multi-worker faults, abandoned-worker bypass — all exit 70, exactly two diagnostic lines, empty stdout).
  • scripts/test-cli.ts locks the negative contract only (a failing file still exits 1 with an unchanged report and no Integrity fault: text).

Expected behavior

  • A build define compiles a fault-injection entry point into the runner (its only purpose: raise an EObjectCheck-class fault inside a chosen test file's execution). The define is set by no shipped build target; the production binary is byte-unaffected.
  • A CI job builds the injection variant and asserts the abort contract: exit 70, exactly two diagnostic lines naming the faulting file, zero summary/envelope output, across --jobs=1/--jobs=N, both --modes, --output=json, and a multi-fault shape (single diagnostic via the first-fault gate).
  • While in the file: the two refusal-family tests in source/units/Goccia.MemoryLimit.Test.pas still carrying the dead caught variable adopt the witness assertion the integrity tests use (one line each; the witness native is already injected).

Scope notes

  • In scope: the deterministic abort contract. Out of scope: the stray-line/truncation race regressions fixed on stack #1159 — the diagnosis proved that window is a Heisenbug (an observation probe made it stop reproducing), so its evidence remains the recorded loop transcript, not a CI assertion.
  • Constraint: the hook must be provably absent from non-injection builds (e.g. a strings/symbol check in the CI job, or the define simply not existing in any build.pas target).
  • Related: ADR 0109 (host tier + seam finding), stack #1159 (#1163, #1164, #1165), docs/testing.md "Engine-integrity faults abort the run".

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with source/app/GocciaTestRunner.dpr and the integrity-abort contract described in ADR 0109 and docs/testing.md. Review scripts/test-cli.ts and source/units/Goccia.MemoryLimit.Test.pas, then inspect the existing CI targets for special-purpose builds. Done means the define-gated variant exercises the stated mode, jobs, output, and multi-fault matrix while non-injection builds exclude the hook and the dead caught variables are replaced with witness assertions.

Written by the indexing model from the issue text.

Assessment

Domain
ci-cd, devtools, testing-qa
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.