rocm serve owner/repo:variant on lemonade exits 0 but never serves the model on Strix Halo Windows
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 40
- Forks
- 9
- Avg merge
- 4d 20h
- Merged PRs (30d)
- 59
Description
Splitting the Windows half out of #247, which noted it "may be two problems rather
than one". The two halves have different symptoms and different failure points, so
they need separate owners. #247 tracks the Linux half; this issue is Windows only.
Symptom
On Windows, serving a Hugging Face GGUF checkpoint by its canonical
owner/repo:variant reference on the lemonade engine appears to succeed and then
never produces a usable endpoint:
rocm serve unsloth/Qwen3-0.6B-GGUF:Q4_0 --engine lemonade --managed
The command exits 0 and reports managed service launched ... readiness: starting.
The model never appears at http://127.0.0.1:11435/v1/models, and after 600 s the
E2E step gives up:
Step panicked. Captured output: endpoint http://127.0.0.1:11435/v1/models
did not serve model Qwen3-0.6B within 600s
Nothing tells the user, or CI, why. A user hitting this sees a serve that claims to
have started and an endpoint that never answers.
What this is not
- Not a broken host or a flake. In the same job on the same machine, scenario 7
serves the same underlying checkpoint successfully via the short recipe name
(Qwen3-0.6B-GGUF), and the ROCm llama.cpp backend installs cleanly
(llama-b9752-bin-win-rocm-7.13-x64.zip+therock-dist-windows-gfx1151-7.13.0.tar.gz,
"Backend installed successfully"). 31 of 35 scenarios pass. Re-running does not help —
the runs below are already attempts 2 and 3. - Not either of the Linux causes. The MI300X lane has two stacked blockers, both
Linux-specific: the Lemonade embeddable's glibc floor (#258) and, once that is
out of the way, an unresolvable runtime directory (#259). Windows uses a
different artifact and a different launch path, and its symptom is the opposite one —
exit 0, not exit 1. - Not platform-wide breakage. Strix Halo Ubuntu passes this scenario on the same
ROCm 7.13.0 / lemonade 11.5.1 pin. - Not a model-id mismatch in the readiness probe. The direct path passes
--alias unsloth/Qwen3-0.6B-GGUF:Q4_0, which contains theQwen3-0.6Bsubstring the
probe looks for, so a live server would match.
Where it likely lives
The two paths diverge in the lemonade engine:
- the short recipe name is served through
lemond; owner/repo:varianttakesserve_hf_checkpoint(engines/lemonade/src/lib.rs:720),
which useslemondonly to pull the GGUF and then launchesllama-serverdirectly
viaserve_direct_llama_server.
Only the second path fails, on Windows only. That narrows it to direct llama-server
launch, GGUF path resolution in the HF cache, or port binding — but which one is not
currently observable (below).
--managed is what makes the exit code useless here: rocm serve returns as soon as the
service is launched, so anything that goes wrong afterwards lands in the managed-service
log, not in the caller's exit status.
Observability gap — please fix as part of this
The failure is currently undiagnosable from CI, and the same gap hurts users:
- No serve output is captured. The step uses the plain
wait_for_model
(tests/e2e-cucumber/tests/e2e/serving_steps.rs:70), which panics with only the
timeout line. Compare the GPU serve step (setup_gpu_model, same file), which on
failure dumps stdout, stderr, the service-log tail and device state. The failing
world dump confirms the loss:cli_output: None, cli_stderr: None, cli_rc: None. - The artifact does not have it either.
record_command
(tests/e2e-cucumber/tests/e2e.rs:528) writes onlyargv/rc/subcommand/command/
model/enginetocommands.jsonl— no stdout, no stderr. - The managed-service log is never uploaded. It is written inside the scenario's
isolated temp data dir (.../data/services/<service-id>.log), which is deleted with
theTempDir; the artifact upload only takestests/e2e-cucumber/results/.
So the one file that would contain the answer is created and destroyed inside the job.
Giving this step the same diagnostics the GPU serve step already collects — including
the managed-service log tail — is the prerequisite for root-causing it. (#248 fixes the
sibling problem of discarded stderr on non-zero exit; it does not cover this path,
which exits 0.)
Beyond CI: a --managed serve that reports success and then silently dies gives the user
nothing to act on either. Surfacing the service log location, or a readiness failure, in
the CLI itself is worth considering under the same fix.
Reproduction
Scenario serve-hf-checkpoint-inference
(tests/e2e-cucumber/features/model_serving.feature:79,
@requires-gpu @requires-engine:lemonade), added by 37a3b63c (#242). It has no xfail row
in tests/e2e-cucumber/expectations.toml, so it is expected to pass on any matching host.
Observed identically on two separate runs of the Strix Halo Windows lane:
- https://github.com/ROCm/rocm-cli/actions/runs/31700949566/job/94449990898 —
35 scenarios (31 passed, 4 failed),3 xfail, 0 XPASS, 1 unexpected failure(s) - https://github.com/ROCm/rocm-cli/actions/runs/31706816911/job/94711942748 — same totals
Environment
- Host: self-hosted Strix Halo Windows runner,
gfx1151, Windows 10.0.26200.9168 - Runtime:
release-wheel-gfx1151-7-13-0; lemonade pinned to11.5.1(runtime-deps.toml) - Workflow:
.github/workflows/e2e-selfhosted.yml, lane "E2E tests (Strix Halo, Windows)" - Model:
unsloth/Qwen3-0.6B-GGUF:Q4_0
Impact
Users on Windows cannot serve a checkpoint by owner/repo:variant on lemonade, and get no
error explaining why. In CI the lane is continue-on-error and not required, so merges are
not blocked; the cost is the same loss of signal described in #247.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with engines/lemonade/src/lib.rs:720 and compare the direct checkpoint path with the existing diagnostics in tests/e2e-cucumber/tests/e2e/serving_steps.rs:70 and setup_gpu_model. Read record_command in tests/e2e-cucumber/tests/e2e.rs:528 and reproduce scenario serve-hf-checkpoint-inference from tests/e2e-cucumber/features/model_serving.feature:79. Done means the Windows failure exposes useful serve output, stderr, and the managed-service log tail instead of only timing out.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- cli, observability-sre, testing-qa
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100