ROCm / ROCm/rocm-cli

rocm serve --engine lemonade: interactive variant selection for owner/repo

Open
#82 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Rust
Stars
40
Forks
9
Avg merge
4d 20h
Merged PRs (30d)
59

Description

Motivation

rocm serve --engine lemonade <owner/repo:variant> serves a Hugging Face GGUF under its canonical name. Today a variant is required — a bare owner/repo, or a partial/ambiguous variant like :Q4 that matches several quantizations, is rejected with an actionable error rather than guessed (to avoid silently serving the wrong quantization). See PR #80.

A nicer UX would be to enumerate the available variants and let the user pick when the variant is missing or ambiguous, instead of erroring.

Proposed approach

  • Enumerate a repo's GGUF variants without downloading (Lemonade exposes a variants endpoint used by its own pull flow).
  • In the foreground/interactive serve path (which has a TTY, during the serve-plan phase in apps/rocm), present the variants and let the user choose, then proceed with the chosen owner/repo:variant.
  • Keep the current non-interactive behavior as the fallback: with no TTY (managed/background serves, scripts, CI), list the available variants in the error message so the user can specify one.

Notes / constraints

  • Interactive selection only applies to the foreground path; managed/background serves must keep the clear error.
  • Belongs in the apps/rocm foreground layer (TTY + serve plan), not the engine's detached serve_http.
  • Invariant to preserve: never silently pick a variant.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in the apps/rocm foreground serve-plan layer and trace the existing variant validation and TTY handling. Inspect the variants endpoint used by Lemonade's pull flow, then define the interactive and no-TTY paths. Done means ambiguous or missing variants can be selected interactively in the foreground, while managed and non-interactive serves list choices without silently selecting one.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
ai, cli
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.