docker / docker/model-runner

compose up: --backend flag is accepted but never applied

Open
#1,036 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Go
Stars
651
Forks
155
PR merge metrics
No merged PRs in 30d

Description

Bug

docker model compose up defines a --backend flag (default llama.cpp) but never uses the value when calling ConfigureBackend.

Evidence (main)

In cmd/cli/commands/compose.go (newUpCommand):

var backend string
// ...
c.Flags().StringVar(&backend, "backend", llamacpp.Name, "inference backend to use")

The only write is the flag binding. backend is never read. Configure is built as:

desktopClient.ConfigureBackend(scheduling.ConfigureRequest{
    Model:                model,
    BackendConfiguration: backendConfig,
    RawRuntimeFlags:      rawRuntimeFlags,
})

There is no backend name field set from the flag. scheduling.ConfigureRequest also has no top-level backend selector (only model + BackendConfiguration + runtime flags). So even if the flag were plumbed, the request shape would need a place to put it (for example how ModelConfigEntry.Backend is stored elsewhere).

Meanwhile compose metadata advertises --backend via commandParameters(upCmd), so Compose/UI consumers may show a backend selector that does nothing.

Expected

Either:

  1. Plumb --backend into the configure/install path so non-default backends work under compose, or
  2. Remove the flag from up and from metadata until the API can honor it (so metadata matches behaviour).

Impact

Users (or Compose integrations) that pass --backend vllm (or anything other than the default) get silent no-ops: the flag is accepted, metadata lists it, but configuration always follows the runner default.

Related

  • Compose only otherwise configures context-size / runtime-flags / speculative fields (#1035 covers env separately).

I can send a PR either wiring backend selection or dropping the dead flag after maintainers pick a direction.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in cmd/cli/commands/compose.go at newUpCommand, then inspect scheduling.ConfigureRequest and the existing ModelConfigEntry.Backend storage mentioned in the issue. Confirm with maintainers whether to wire the flag through the configure/install path or remove it from up and commandParameters; done means non-default backends work or the flag and metadata no longer advertise unsupported behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
docker, go
Domain
backend, cli
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.