mudler / mudler/vllm.cpp

test_cpu_x86_llamacpp_floor reds as a function of machine load, on branches that never touched it

Open Beginner friendly
#2,976 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
C++
Stars
423
Forks
53
Avg merge
20h 26m
Merged PRs (30d)
310

Description

Row: BACKEND-GATE-CPU-LLAMACPP

What

tests/scripts/test_cpu_x86_llamacpp_floor.py's
test_a_contended_leg_is_discarded_and_never_summarised (line 163) reds as a
function of MACHINE LOAD rather than of the tree. It is in agent-preflight.sh,
so it reds an arbitrary unrelated branch whenever the box is busy.

self.assertEqual(got.returncode, 2, got.stdout + got.stderr)

Measured, three times, on two different trees

When Load Actual rc Harness said
2026-09-05 ~02:20 78.6 4 NO_QUIET_WINDOW after 30s (busy=116% builders=0)
2026-09-05 ~21:34 46.0 4 ours rep=1 START load=46.01 43.25 35.99 builders=0
2026-09-05 ~21:37 26.5 0 (OK) 10 tests passed

Same commit, same binary, three verdicts. It also passed on an unmodified
origin/main checkout at load 72 earlier the same day, so it is not monotone in
load either — it is a race against whether a quiet window appears.

The actual defect

The case asserts the harness reaches its discard-a-contended-leg path and
exits 2. Under real contention the harness never gets that far: it fails earlier,
at acquiring a quiet window at all, and exits 4. Both are correct harness
behaviours. The test pins one of them and gets the other whenever the box is
loaded enough to trigger the earlier guard first.

So the test conflates two exit paths, and which one it observes is decided by the
machine, not by the code under test.

Why it is worth fixing rather than tolerating

AGENTS.md: "A gate that fires on ordinary work is the defect, not the
discipline." This one has cost three preflight runs across two sessions, and each
red was charged to a branch that had not touched the harness — the exact pattern
#2950 is about.

It is also the worst kind of flake to leave: it fails on a BUSY box, which is
precisely when an agent is most likely to be mid-campaign and least likely to
have spare attention to attribute it.

Suggested shape, not prescribed

Accept either terminal path for a contended run — assert rc in (2, 4) and that
NO summary was produced in both — or drive the discard path with a synthetic
contention signal instead of real load, so the case tests the harness's logic
rather than the box's mood. Whichever is chosen, it must not weaken what the case
actually guards: that a contended leg is never summarised.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with tests/scripts/test_cpu_x86_llamacpp_floor.py, especially test_a_contended_leg_is_discarded_and_never_summarised at line 163, and inspect the agent-preflight.sh paths that return 2 or 4. Run the targeted test and choose either accepting both terminal paths or using synthetic contention; done means a contended leg is never summarised without depending on machine load.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, shell
Domain
ci-cd, testing
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
76/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.