frostney / frostney/GocciaScript
Reject focused tests in continuous integration
Nobody has claimed this yet.
- Dominant language
- Pascal
- Stars
- 20
- Forks
- 3
- Avg merge
- 3d 4h
- Merged PRs (30d)
- 45
Description
Summary
Make GocciaTestRunner reject focused test.only / describe.only registrations by default when it detects a continuous-integration environment.
Why
Focused tests are useful during local development, but a committed .only can silently turn a full CI run into a passing subset. CI should fail loudly instead of producing a false-green result.
Current behavior
GocciaTestRunner already detects common CI environments in IsContinuousIntegration, but uses that signal only to disable snapshot updates. Focused registrations are selected in Goccia.Builtins.TestingLibrary.pas without a CI guard.
A current-main reproduction exits successfully while skipping the ordinary test:
printf '%s\n' 'test("ordinary", () => { expect(1).toBe(1); }); test.only("focused", () => { expect(1).toBe(1); });' | CI=true ./build/GocciaTestRunner - --no-progress --output=json
The result reports two registered tests, one passed test, one skipped test, and exit status 0.
Expected behavior
- In a detected CI environment, fail with a non-zero exit status when any focused test or suite is registered.
- Identify the focused test/suite names and source files in the diagnostic.
- Apply the guard in interpreter and bytecode modes, including serial and
--jobsruns. - Provide an explicit
--allow-onlyescape hatch for intentional CI diagnostics.
Scope notes
- Local
.onlybehavior remains unchanged unless CI is detected. - Do not add a separate source linter or broaden CI-provider detection beyond the existing helper as part of this issue.
- This closes a runner-policy gap left by the focused-registration work in #118 and the parent API issue #114.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with GocciaTestRunner.IsContinuousIntegration and the focused-registration logic in Goccia.Builtins.TestingLibrary.pas. Run the CI=true reproduction from the issue, then check interpreter, bytecode, serial, and --jobs paths. Done means focused registrations fail CI with names and source files, while --allow-only permits intentional runs and local behavior remains unchanged.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- javascript
- Domain
- ci-cd, cli, testing
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100