test_cpu_x86_llamacpp_floor reds as a function of machine load, on branches that never touched it
Nobody has claimed this yet.
- Dominant language
- C++
- Stars
- 423
- Forks
- 53
- Avg merge
- 20h 26m
- Merged PRs (30d)
- 310
Description
Row: BACKEND-GATE-CPU-LLAMACPP
What
tests/scripts/test_cpu_x86_llamacpp_floor.py's
test_a_contended_leg_is_discarded_and_never_summarised (line 163) reds as a
function of MACHINE LOAD rather than of the tree. It is in agent-preflight.sh,
so it reds an arbitrary unrelated branch whenever the box is busy.
self.assertEqual(got.returncode, 2, got.stdout + got.stderr)
Measured, three times, on two different trees
| When | Load | Actual rc | Harness said |
|---|---|---|---|
| 2026-09-05 ~02:20 | 78.6 | 4 | NO_QUIET_WINDOW after 30s (busy=116% builders=0) |
| 2026-09-05 ~21:34 | 46.0 | 4 | ours rep=1 START load=46.01 43.25 35.99 builders=0 |
| 2026-09-05 ~21:37 | 26.5 | 0 (OK) | 10 tests passed |
Same commit, same binary, three verdicts. It also passed on an unmodified
origin/main checkout at load 72 earlier the same day, so it is not monotone in
load either — it is a race against whether a quiet window appears.
The actual defect
The case asserts the harness reaches its discard-a-contended-leg path and
exits 2. Under real contention the harness never gets that far: it fails earlier,
at acquiring a quiet window at all, and exits 4. Both are correct harness
behaviours. The test pins one of them and gets the other whenever the box is
loaded enough to trigger the earlier guard first.
So the test conflates two exit paths, and which one it observes is decided by the
machine, not by the code under test.
Why it is worth fixing rather than tolerating
AGENTS.md: "A gate that fires on ordinary work is the defect, not the
discipline." This one has cost three preflight runs across two sessions, and each
red was charged to a branch that had not touched the harness — the exact pattern
#2950 is about.
It is also the worst kind of flake to leave: it fails on a BUSY box, which is
precisely when an agent is most likely to be mid-campaign and least likely to
have spare attention to attribute it.
Suggested shape, not prescribed
Accept either terminal path for a contended run — assert rc in (2, 4) and that
NO summary was produced in both — or drive the discard path with a synthetic
contention signal instead of real load, so the case tests the harness's logic
rather than the box's mood. Whichever is chosen, it must not weaken what the case
actually guards: that a contended leg is never summarised.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with tests/scripts/test_cpu_x86_llamacpp_floor.py, especially test_a_contended_leg_is_discarded_and_never_summarised at line 163, and inspect the agent-preflight.sh paths that return 2 or 4. Run the targeted test and choose either accepting both terminal paths or using synthetic contention; done means a contended leg is never summarised without depending on machine load.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, shell
- Domain
- ci-cd, testing
- Issue type
- Bug
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 76/100