Server silently serves the resident model for unresolvable model tags, echoing the requested name
Nobody has claimed this yet.
- Dominant language
- C++
- Stars
- 1.9k
- Forks
- 152
- Avg merge
- 4h 14m
- Merged PRs (30d)
- 11
Description
Summary
flm serve answers requests for model tags it cannot resolve by running whatever model is
currently loaded, and echoes the requested tag back in the response model field. Nothing in
the response or the server log indicates the substitution. A benchmark script can therefore collect
timings for a model that never loaded.
Reproduce (10 seconds)
flm serve qwen3:1.7b --host 127.0.0.1 --port 8093
curl -s http://127.0.0.1:8093/v1/chat/completions -H 'Content-Type: application/json' \
-d '{"model":"totally-made-up:99b","messages":[{"role":"user","content":"Name three colors."}],"max_tokens":40}'
Actual — a fully formed completion for a tag that exists nowhere:
{"model":"totally-made-up:99b",
"choices":[{"message":{"role":"assistant","content":"Here are three colors:\n\n1. Red\n2. Blue\n3. Green"}}],
"usage":{"decoding_speed_tps":43.27, ...}}
Expected: an error naming the unresolvable tag.
Same on /v1/audio/transcriptions: one loaded Whisper answered to whisper-v3:turbo,
whisper-v3-turbo-FLM and whisper-1 alike, each echoed back verbatim.
Why it matters
The model field is the only signal a client has about which model ran, and it is an echo of the
request rather than a statement of fact. The failure is easiest to hit exactly when a tag should
fail — mid-setup, before model_list.json is correct — which is also when people run their first
benchmark. The result looks completely normal.
Note this is distinct from a registered-but-unloadable tag, which does error correctly.
Suggested fix
Return an error for an unresolvable tag instead of falling through to the resident model. Failing
that, populate the response model field with the model that actually served the request, so the
output is self-describing.
Environment: FLM v1.0.1 / v1.0.2 / v1.0.4, Ryzen AI Max+ 395, NPU fw 1.1.2.65, in-tree
amdxdna 0.7, kernel 7.0.0-31-generic, Ubuntu 26.04.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start at the HTTP handling path used by flm serve for /v1/chat/completions and /v1/audio/transcriptions, then trace how requested model tags are resolved before inference. Reproduce the fallback with the provided curl request and verify that an unresolvable tag returns an error rather than using the resident model; the response should no longer claim the requested tag ran.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- api, backend
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 64/100