mudler / mudler/vllm.cpp

ENG-DOCS-SITE cites six line anchors into test_check_site.py that land on no test, and undercounts the cases

Open
#2,678 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
C++
Stars
423
Forks
53
Avg merge
20h 26m
Merged PRs (30d)
310

Description

Row: ENG-DOCS-SITE

Found while repairing #2643. Not caused by it, and deliberately not repaired
there: see below for why the obvious repair is a design decision this row's
owner should make.

The drift

.agents/engine-matrix.md, the ENG-DOCS-SITE row, cites its test evidence as

`tests/scripts/test_check_site.py:51,56,66,80,89,97` (6 mutation cases: clean
tree, H1 stripped, doc absent from nav, nav entry with no file, duplicated
entry, missing nav file)

None of those six lines lands on a def test_. Measured at 39757a0c7, before
any change of mine:

$ git show 39757a0c7:tests/scripts/test_check_site.py | sed -n '51p;56p;66p;80p;89p;97p'
    )
        tmp = Path(tempfile.mkdtemp())
            shutil.copy(doc, tree / "docs" / doc.name)
        tree = self.scratch()
    def test_a_doc_missing_from_nav_is_caught(self) -> None:
        nav.write_text("".join(kept))

The six cases were at 74, 79, 89, 103, 112 and 120 at that commit. One of the
six cited lines happens to land on a def test_, and it is the wrong one for
its position in the list.

The cell also says "6 mutation cases". There are seven; the seventh,
test_rendered_benchmark_index_links_resolve_to_emitted_pages, arrived with
#1714 and was never added here.

Why #2643 did not repair it

The repair of #2643 adds a module-level helper to that file and moves every case
down by 21 lines, so it makes an already-wrong citation differently wrong. It
does not repair it, for two reasons.

  1. The obvious fix interacts with a gate. Citing the cases by name instead
    of by line is the fix that cannot rot again, and it is what this repository's
    own experience argues for. But is_test_anchor in
    scripts/check-agent-record.py recognises a tests cell only through
    local_line_anchors, which requires a path:line that resolves and is in
    range. Dropping the numbers would make the cell carry no test anchor at all.
    Whether that matters, and what the cell should look like instead, is a
    decision about the anchor contract and not about the docs site.
  2. .agents/engine-matrix.md is a keyed record on a shared surface. Two
    open pull requests already write it. Smuggling an unreviewed edit to another
    row into a gate-repair pull request is how these records get merged into a
    state nobody chose.

What is not affected

The record-anchor ratchet does not see this. ANCHOR-ROT reads 33 before and
after, matching scripts/record-anchor-baseline.json, because the ratchet
reports a citation as STALE only when a symbol is named beside the line, and
this cell names none. So this is unread prose today, which is the reason it
could rot in silence and the reason no gate will catch the next drift either.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the ENG-DOCS-SITE row in .agents/engine-matrix.md and compare its cited anchors with the test definitions in tests/scripts/test_check_site.py. Then read is_test_anchor and local_line_anchors in scripts/check-agent-record.py, along with the open pull requests touching the shared matrix. Done means the anchor contract and the complete set of seven cases are explicitly agreed and recorded without an unrelated row edit.

Written by the indexing model from the issue text.

Assessment

Tech stack
markdown, python
Domain
documentation, testing-qa, tooling
Issue type
Documentation
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.