CI's zero-warning budget is Linux-only; 2,700+ core tests are compiled but never run
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 19.9k
- Forks
- 2.3k
- Avg merge
- 2d 7h
- Merged PRs (30d)
- 30
Description
Summary
The zero-warning budget and most --lib test suites are only enforced on Linux, so two classes of defect reach master with CI fully green:
- macOS-only compiler warnings.
cargo check -qon macOSmaster(ff6fb6359) emits 5 warnings. The budget inscripts/warning_budget.txtis0. - Unrun tests that are already failing.
jcode-core --libhas a test that fails on cleanmasteron macOS and is never executed by any CI job.
These are the same shape as #651, #660, and #592, which the workflow comments already document. This issue is about the general gate rather than another instance.
Evidence 1: macOS warnings on clean master
$ git checkout ff6fb6359
$ CARGO_TERM_COLOR=never cargo check -q 2>&1 | grep -c '^warning:'
5
They are:
warning: function `is_file_controlled_debug_client` is never used
warning: function `proc_pidfdinfo` is never used
warning: constant `PROC_PIDFDVNODEPATHINFO` is never used
warning: constant `PROC_PIDFDSOCKETINFO` is never used
warning: constant `PROC_PIDFDPIPEINFO` is never used
is_file_controlled_debug_client (src/cli/dispatch.rs:24) is only consumed by arm_debug_client_parent_death_signal, which is #[cfg(target_os = "linux")]. The others are macOS proc_pidfdinfo FFI declarations in crates/jcode-core/src/stdin_detect.rs that nothing references.
CI cannot see any of these:
.github/workflows/ci.yml— thequalityjob runs onubuntu-latestonly.- The
Enforce warning budget (Linux)step inbuildisif: runner.os == 'Linux'.
So the macOS runner builds this code and never counts its warnings.
Evidence 2: a test that is red on master and never runs
$ git checkout ff6fb6359 # clean upstream, no local patches
$ cargo test -p jcode-core --lib stdin
---- stdin_detect::stdin_detect_tests::test_own_process_not_reading_stdin stdout ----
thread '...' panicked at crates/jcode-core/src/stdin_detect_tests.rs:9:5:
assertion `left != right` failed
left: Reading
right: Reading
test result: FAILED. 1 passed; 1 failed
Verified in a clean git worktree of origin/master, so this is not a local artifact.
The test asserts the current process is not reading stdin. On macOS it is reported as Reading, which is the false positive tracked in #1146. The failing test is therefore an existing, checked-in reproduction of #1146 that nobody sees.
jcode-core --lib is not run by any job. The only stdin-related CI step is:
-p jcode-app-core --lib tool::bash::tests::test_stdin_forwarding
which is a different crate and a different code path.
Scale of the unrun surface
Tests that exist but are not executed by any CI job:
| Crate | #[test] / #[tokio::test] |
|---|---|
jcode-base |
1,377 |
jcode-app-core |
1,335 |
jcode-core |
39 |
jcode-app-core and jcode-base are compiled via --lib --bins --no-run and then only four filtered cohorts are actually run (retention_readiness, secret_input, test_stdin_forwarding, plus the embedding cohort). jcode-tui --lib is the only broad suite that runs, and it runs serially.
Suggested direction
Two independent changes; the first is nearly free.
1. Enforce the warning budget on macOS. Drop the if: runner.os == 'Linux' condition on the warning-budget step in the build matrix job. This requires the 5 warnings above to be fixed first, otherwise the macOS leg goes red immediately.
I have that part done and tested on ianalitis:chore/macos-warning-clean (3 commits, macOS cargo check -q goes 5 warnings -> 0). Happy to open it as a PR if you want it, though I currently cannot create PRs against this repo (see note below).
2. Run the core lib suites, serially. Mirror the existing jcode-tui --lib step for the two big crates:
- name: Run core library tests (Linux only)
if: runner.os == 'Linux'
run: |
python3 .github/scripts/run_with_timeout.py 900 \
"$(rustup which cargo)" test --target ${{ matrix.target }} \
-p jcode-app-core -p jcode-base -p jcode-core --lib -- --test-threads=1
Serial execution is required for the same reason it is on jcode-tui: shared process-global state (#592, #1141). I expect this to be red on the first run. The value is that the resulting list becomes a visible backlog rather than a silent one.
If a fully green gate is wanted immediately, this could start as a separate non-blocking job and be promoted once the list is drained.
Note
I cannot open pull requests against this repository with my current token (createPullRequest returns 404 / "does not have the correct permissions"), though issue creation works. The branch above is pushed to my fork if you would like to pull from it, or I am happy to attach the diffs here instead.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with .github/workflows/ci.yml, scripts/warning_budget.txt, and the existing Linux warning-budget and jcode-tui test steps. Run cargo check -q and the affected jcode-core library test on macOS, then inspect the three listed library test suites and their current CI coverage. Done means the intended warning budget and library-test coverage are enforced without masking the known failures.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- github-actions, rust
- Domain
- ci-cd, testing-qa
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100