ROCm / ROCm/FastFlowLM

Getting flm server to play ball with Codex?

Open
#615 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
C++
Stars
1.9k
Forks
152
Avg merge
4h 14m
Merged PRs (30d)
11

Description

I have one of these AMD-based NPUs and using Windows, and I wanted to see how well this NPU-based implementation works. This is probably a skill issue, but I am struggling to get flm to work with Codex.

The first weird thing is the Codex ignored my set environment variables and it defaulted to OpenAI servers. OK, I can live with that. I have set these as:

Get-ChildItem Env:

OPENAI_API_BASE                http://localhost:52625
OPENAI_API_KEY                 flm

So I changed the config file and created my custom profile. Here is the reelevant part of .codex\config.toml

model = "qwen3.6-moe:35b-a3b"
model_provider = "flm"

[model_providers.flm]
name = "FastFlowLM"
base_url = "http://localhost:52625/v1"
env_key = "OPENAI_API_KEY"
wire_api = "responses"

In one terminal window, I run flm serve qwen3.6-moe:35b-a3b --ctx-len 262144. then, in another terminal window, I run: codex --profile flm

Codex loads, I send in a prompt, the model loads, and I can see the output of the server.

But, I don't actually get a response in Codex, but a warning and an error message:

⚠ Model metadata for `qwen3.6-moe:35b-a3b` not found. Defaulting to fallback metadata; this can degrade performance and cause issues.

■ unexpected status 404 Not Found: {"error":"Not Found"}, url: http://localhost:52625/v1/responses

So, can anyone tell me what am I doing wrong?

(edit: codex CLI no longer supports 'chat' and 'completions', I have to use 'responses', and I see in the documentation that 'responses' endpoint is not supported - so probably that's it)

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the Windows setup with flm serve qwen3.6-moe:35b-a3b --ctx-len 262144 and the supplied .codex\config.toml. Check the documented API endpoints against Codex's request to /v1/responses; done means the supported integration path is identified and either works or the incompatibility is clearly documented.

Written by the indexing model from the issue text.

Assessment

Tech stack
powershell
Domain
api, backend-api-design
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.