rocm serve --engine lemonade: interactive variant selection for owner/repo
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 40
- Forks
- 9
- Avg merge
- 4d 20h
- Merged PRs (30d)
- 59
Description
Motivation
rocm serve --engine lemonade <owner/repo:variant> serves a Hugging Face GGUF under its canonical name. Today a variant is required — a bare owner/repo, or a partial/ambiguous variant like :Q4 that matches several quantizations, is rejected with an actionable error rather than guessed (to avoid silently serving the wrong quantization). See PR #80.
A nicer UX would be to enumerate the available variants and let the user pick when the variant is missing or ambiguous, instead of erroring.
Proposed approach
- Enumerate a repo's GGUF variants without downloading (Lemonade exposes a variants endpoint used by its own
pullflow). - In the foreground/interactive serve path (which has a TTY, during the serve-plan phase in
apps/rocm), present the variants and let the user choose, then proceed with the chosenowner/repo:variant. - Keep the current non-interactive behavior as the fallback: with no TTY (managed/background serves, scripts, CI), list the available variants in the error message so the user can specify one.
Notes / constraints
- Interactive selection only applies to the foreground path; managed/background serves must keep the clear error.
- Belongs in the
apps/rocmforeground layer (TTY + serve plan), not the engine's detachedserve_http. - Invariant to preserve: never silently pick a variant.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in the apps/rocm foreground serve-plan layer and trace the existing variant validation and TTY handling. Inspect the variants endpoint used by Lemonade's pull flow, then define the interactive and no-TTY paths. Done means ambiguous or missing variants can be selected interactively in the foreground, while managed and non-interactive serves list choices without silently selecting one.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- ai, cli
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 55/100