gpustack / gpustack/gpustack-operator

todo: the cross-vendor KV verification matrix, and what each row does not answer

Open
#455 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

area/testing todo
Dominant language
Go
Stars
4
Forks
7
Avg merge
3h 9m
Merged PRs (30d)
213

Description

One place to see every cross-vendor and Ascend verification this project owes, what hardware each row
needs, and — for each row — what does not answer it.

This consolidates #216 and #333, which were separate carriers for rows of one matrix. Their
"what does not close this" paragraphs are reproduced verbatim below, because those paragraphs are
what the issues were actually worth: each names the observation that looks like success and is not.

The rows are not interchangeable and two of them are not even about two vendors. Reading a green
result on one row as an answer to another is the failure this issue exists to prevent, so each row
states its own question.

The matrix

# question vendors hardware state
R1 Does an Ascend engine actually serve with the configuration this operator injects? Ascend alone one Ascend node unverified — was #333
R2 Can an -npu client and a -cuda13 client share one master and read each other's segments? Ascend + NVIDIA Ascend node and NVIDIA node, one cluster unverified — was #216
R3 What does this operator render for a cross-manufacturer pair? any two none — fixtures only tracked in #435, item 3

R1 blocks R2. A single Ascend engine that has not been shown to serve with our injected
configuration cannot be one half of a mutual-read test: a failure would not distinguish "the two
halves cannot share" from "that half was never serving".

R3 is a different layer and does not block either. It answers what this operator writes into an
engine's configuration. Whether the two halves can then share anything is not something a fixture
reaches.

R1 — an Ascend engine serving with the configuration we inject

From #333. case-60's vLLM-Ascend row asks two questions and its header separates them:

  1. "Is the name we render in that engine's registry?" — ANSWERED, and answered better without a
    cluster: one docker run --rm per image, listing the whole registry rather than probing for one
    name. Verbatim: "Re-running it through a Pod adds no information."
  2. "Does the engine actually run with the configuration we inject?" — NOT ANSWERABLE on a machine
    with no accelerator, structurally. Measured: vllm-ascend aborts on torch_npu before any vLLM
    module loads.

Only the second is the gap.

What does NOT close this (verbatim from #333):

  • A cluster run without Ascend hardware, for the three reasons quoted above.
  • Building the second Ascend-transport backend and pool. That is a means; on hardware-less clusters
    it cannot answer question 2, and on real hardware the ModelDeployment path answers it more
    directly.
  • Re-running the registry listing. That is question 1, already answered, and answered better offline.
  • Removing the skipped row. The row is the record that question 2 is unverified.

Two pointers still have to move, and neither is done (carried over from #333, unchanged):

  • case-60's header still says "Tracked with the rest of that family in issue #172". That pointer is
    dead and has to name this issue instead.
  • pkg/worker/kvcache/inject/engine.go:198-203 carries a "HOW TO EXERCISE A ROW" note saying the
    vLLM-Ascend transport row is "reachable only through the ModelDeployment path", and warning that
    trying it from a Pod annotation gets refused by ParseEngine "on the engine name and not on the
    transport". Accurate for the vllm-ascend spelling, but it omits the manufacturer spelling
    (engine: vllm + manufacturer: ascend), which pod_kv_cache_inject_test.go:90 shows is admitted
    on an Ascend pool. A reader following that note concludes the annotation route does not exist at
    all.

Until those move, a reader arriving through the code still lands on a closed issue.

R2 — heterogeneous clients sharing one master

From #216. Layer: L2 · Source: S2.

Whether heterogeneous clients — the -npu build and the -cuda13 build — can share one master and
read each other's segments. S2's own words: "Verified only as loading."

What does NOT fill this (verbatim from #216):

Both images starting does not count. Loading is what has already been verified; the assertion
here is mutual reads — one build writing a segment the other build can read back.

Hardware: same set of machines as the Ascend work. Stating the requirement in those same words on
purpose, so the two are recognisable as one hardware ask rather than two.

R3 — what the operator renders for a cross-manufacturer pair

Stays in #435 as its item 3, alongside the two other items of that issue. Listed here so this matrix
is complete, not to move it. It is fixture-only: no hardware, no cluster, four independent
observables (admission, entrance and credits, image backend, connector name) which can disagree with
each other, which is why they are kept apart.

What a reader should not conclude from this issue

  • That a green run on R1 says anything about R2. R1 is one vendor.
  • That R3 passing means cross-manufacturer P/D works. The fixtures answer what this operator writes
    into the engine's configuration; whether the two halves can share is a property of the two engine
    projects, and no fixture reaches it.
  • That this issue covers the device-manager side. nodeWideDeviceGroups has never run on a node
    carrying two accelerator manufacturers — that is #181, a different subsystem with a different
    observable, and it is deliberately not a row here.

What does NOT close this issue

Closing any single row. The rows have separate blockers and separate hardware; a row that is answered
gets its answer recorded here and struck from the table, and the issue stays open while any row is
unanswered.

/kind cleanup
/area testing

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the R1 references in case-60, pkg/worker/kvcache/inject/engine.go:198-203, and pod_kv_cache_inject_test.go:90, then review the R2 and R3 pointers. Run the named tests where hardware permits and record each row's distinct result. Done means the stale pointers and exercise note are corrected and every matrix row has an explicit answer or remains clearly marked unverified.

Written by the indexing model from the issue text.

Assessment

Tech stack
docker, go, kubernetes
Domain
devops, infrastructure, testing-qa
Issue type
Documentation
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.