frostney / frostney/GocciaScript

Reject focused tests in continuous integration

Open
#1,035 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

internal new feature runtime
Dominant language
Pascal
Stars
20
Forks
3
Avg merge
3d 4h
Merged PRs (30d)
45

Description

Summary

Make GocciaTestRunner reject focused test.only / describe.only registrations by default when it detects a continuous-integration environment.

Why

Focused tests are useful during local development, but a committed .only can silently turn a full CI run into a passing subset. CI should fail loudly instead of producing a false-green result.

Current behavior

GocciaTestRunner already detects common CI environments in IsContinuousIntegration, but uses that signal only to disable snapshot updates. Focused registrations are selected in Goccia.Builtins.TestingLibrary.pas without a CI guard.

A current-main reproduction exits successfully while skipping the ordinary test:

printf '%s\n' 'test("ordinary", () => { expect(1).toBe(1); }); test.only("focused", () => { expect(1).toBe(1); });' | CI=true ./build/GocciaTestRunner - --no-progress --output=json

The result reports two registered tests, one passed test, one skipped test, and exit status 0.

Expected behavior

  • In a detected CI environment, fail with a non-zero exit status when any focused test or suite is registered.
  • Identify the focused test/suite names and source files in the diagnostic.
  • Apply the guard in interpreter and bytecode modes, including serial and --jobs runs.
  • Provide an explicit --allow-only escape hatch for intentional CI diagnostics.

Scope notes

  • Local .only behavior remains unchanged unless CI is detected.
  • Do not add a separate source linter or broaden CI-provider detection beyond the existing helper as part of this issue.
  • This closes a runner-policy gap left by the focused-registration work in #118 and the parent API issue #114.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with GocciaTestRunner.IsContinuousIntegration and the focused-registration logic in Goccia.Builtins.TestingLibrary.pas. Run the CI=true reproduction from the issue, then check interpreter, bytecode, serial, and --jobs paths. Done means focused registrations fail CI with names and source files, while --allow-only permits intentional runs and local behavior remains unchanged.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript
Domain
ci-cd, cli, testing
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.