1jehuang / 1jehuang/jcode

CI's zero-warning budget is Linux-only; 2,700+ core tests are compiled but never run

Open
#1,177 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

tech-debt triage: needs-decision
Dominant language
Rust
Stars
19.9k
Forks
2.3k
Avg merge
2d 7h
Merged PRs (30d)
30

Description

Summary

The zero-warning budget and most --lib test suites are only enforced on Linux, so two classes of defect reach master with CI fully green:

  1. macOS-only compiler warnings. cargo check -q on macOS master (ff6fb6359) emits 5 warnings. The budget in scripts/warning_budget.txt is 0.
  2. Unrun tests that are already failing. jcode-core --lib has a test that fails on clean master on macOS and is never executed by any CI job.

These are the same shape as #651, #660, and #592, which the workflow comments already document. This issue is about the general gate rather than another instance.

Evidence 1: macOS warnings on clean master

$ git checkout ff6fb6359
$ CARGO_TERM_COLOR=never cargo check -q 2>&1 | grep -c '^warning:'
5

They are:

warning: function `is_file_controlled_debug_client` is never used
warning: function `proc_pidfdinfo` is never used
warning: constant `PROC_PIDFDVNODEPATHINFO` is never used
warning: constant `PROC_PIDFDSOCKETINFO` is never used
warning: constant `PROC_PIDFDPIPEINFO` is never used

is_file_controlled_debug_client (src/cli/dispatch.rs:24) is only consumed by arm_debug_client_parent_death_signal, which is #[cfg(target_os = "linux")]. The others are macOS proc_pidfdinfo FFI declarations in crates/jcode-core/src/stdin_detect.rs that nothing references.

CI cannot see any of these:

  • .github/workflows/ci.yml — the quality job runs on ubuntu-latest only.
  • The Enforce warning budget (Linux) step in build is if: runner.os == 'Linux'.

So the macOS runner builds this code and never counts its warnings.

Evidence 2: a test that is red on master and never runs

$ git checkout ff6fb6359    # clean upstream, no local patches
$ cargo test -p jcode-core --lib stdin

---- stdin_detect::stdin_detect_tests::test_own_process_not_reading_stdin stdout ----
thread '...' panicked at crates/jcode-core/src/stdin_detect_tests.rs:9:5:
assertion `left != right` failed
  left: Reading
 right: Reading

test result: FAILED. 1 passed; 1 failed

Verified in a clean git worktree of origin/master, so this is not a local artifact.

The test asserts the current process is not reading stdin. On macOS it is reported as Reading, which is the false positive tracked in #1146. The failing test is therefore an existing, checked-in reproduction of #1146 that nobody sees.

jcode-core --lib is not run by any job. The only stdin-related CI step is:

-p jcode-app-core --lib tool::bash::tests::test_stdin_forwarding

which is a different crate and a different code path.

Scale of the unrun surface

Tests that exist but are not executed by any CI job:

Crate #[test] / #[tokio::test]
jcode-base 1,377
jcode-app-core 1,335
jcode-core 39

jcode-app-core and jcode-base are compiled via --lib --bins --no-run and then only four filtered cohorts are actually run (retention_readiness, secret_input, test_stdin_forwarding, plus the embedding cohort). jcode-tui --lib is the only broad suite that runs, and it runs serially.

Suggested direction

Two independent changes; the first is nearly free.

1. Enforce the warning budget on macOS. Drop the if: runner.os == 'Linux' condition on the warning-budget step in the build matrix job. This requires the 5 warnings above to be fixed first, otherwise the macOS leg goes red immediately.

I have that part done and tested on ianalitis:chore/macos-warning-clean (3 commits, macOS cargo check -q goes 5 warnings -> 0). Happy to open it as a PR if you want it, though I currently cannot create PRs against this repo (see note below).

2. Run the core lib suites, serially. Mirror the existing jcode-tui --lib step for the two big crates:

- name: Run core library tests (Linux only)
  if: runner.os == 'Linux'
  run: |
    python3 .github/scripts/run_with_timeout.py 900 \
      "$(rustup which cargo)" test --target ${{ matrix.target }} \
      -p jcode-app-core -p jcode-base -p jcode-core --lib -- --test-threads=1

Serial execution is required for the same reason it is on jcode-tui: shared process-global state (#592, #1141). I expect this to be red on the first run. The value is that the resulting list becomes a visible backlog rather than a silent one.

If a fully green gate is wanted immediately, this could start as a separate non-blocking job and be promoted once the list is drained.

Note

I cannot open pull requests against this repository with my current token (createPullRequest returns 404 / "does not have the correct permissions"), though issue creation works. The branch above is pushed to my fork if you would like to pull from it, or I am happy to attach the diffs here instead.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with .github/workflows/ci.yml, scripts/warning_budget.txt, and the existing Linux warning-budget and jcode-tui test steps. Run cargo check -q and the affected jcode-core library test on macOS, then inspect the three listed library test suites and their current CI coverage. Done means the intended warning budget and library-test coverage are enforced without masking the known failures.

Written by the indexing model from the issue text.

Assessment

Tech stack
github-actions, rust
Domain
ci-cd, testing-qa
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.