ROCm / ROCm/FastFlowLM

/api/show returns stubs — populate model_info context_length + details so Ollama-compatible clients can auto-configure

Open
#671 3 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
C++
Stars
1.9k
Forks
152
Avg merge
4h 14m
Merged PRs (30d)
11

Description

Problem

flm serve exposes Ollama-compatible endpoints, but /api/show returns a hardcoded stub that ignores the requested model (verified on v1.0.1, same for every tag — loaded or not):

{"details": {"family": "", "parameter_size": "", "quantization_level": ""},
 "model_info": {"general.architecture": "flm"},
 "capabilities": ["chat", "vision", "completion"]}

Context length is exposed nowhere in the HTTP API — /v1/models returns only id/owned_by (no meta.n_ctx like llama-server). So Ollama-API clients cannot auto-configure: they get context_window=0 for every model and need hardcoded per-model tables.

Goal

As an Ollama-API client, I query /api/show and get correct context length, parameter size, and quantization per model — no hardcoded tables. E.g. provider add flm --type ollama --base-url http://localhost:52625 in agent CLIs like Crush (reads any *.context_length key from model_info) then works with zero extra config, discovery included.

Evidence (it's wiring, not missing data)

  1. handle_show (src/server/rest_handler.cpp:599-623) returns a hardcoded literal JSON — it reads request["model"] and then ignores it. No lookup is performed.
  2. The registry (model_list.json) already carries everything per model, e.g. qwen3.6-moe:35b-a3b:
    {"default_context_length": 32768, "max_prefill_len": 4096, "vlm": true,
     "details": {"format": "NPU2", "family": "qwen3.6-moe", "parameter_size": "35B",
                 "quantization_level": "Q4_K_S", "think": true}}
    
  3. /api/ps already serializes this same data (model_info["details"], rest_handler.cpp:1008-1015) — and returns real values for the running model, while /api/show returns empty strings for that same model. Only context length is missing from both.

Suggested fix

In handle_show: call supported_models.get_model_info(model), serialize details, and emit flm.context_length (= default_context_length) in model_info. Optionally mirror context in /v1/models.

Open design choice: report default_context_length (what the server actually runs, matches Ollama's n_ctx semantics) vs. the model family max (256k here).

Research was done by GLM-5.3 via Crush

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in src/server/rest_handler.cpp at handle_show (599-623), then inspect supported_models.get_model_info and the per-model entries in model_list.json. Compare the existing serialization in the /api/ps handler (1008-1015), and verify /api/show for a known model and an unavailable model. Done means the response contains that model's details and default_context_length as flm.context_length, with the chosen context-length semantics documented or consistently applied.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
api, backend
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
74/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.