Getting flm server to play ball with Codex?
Nobody has claimed this yet.
- Dominant language
- C++
- Stars
- 1.9k
- Forks
- 152
- Avg merge
- 4h 14m
- Merged PRs (30d)
- 11
Description
I have one of these AMD-based NPUs and using Windows, and I wanted to see how well this NPU-based implementation works. This is probably a skill issue, but I am struggling to get flm to work with Codex.
The first weird thing is the Codex ignored my set environment variables and it defaulted to OpenAI servers. OK, I can live with that. I have set these as:
Get-ChildItem Env:
OPENAI_API_BASE http://localhost:52625
OPENAI_API_KEY flm
So I changed the config file and created my custom profile. Here is the reelevant part of .codex\config.toml
model = "qwen3.6-moe:35b-a3b"
model_provider = "flm"
[model_providers.flm]
name = "FastFlowLM"
base_url = "http://localhost:52625/v1"
env_key = "OPENAI_API_KEY"
wire_api = "responses"
In one terminal window, I run flm serve qwen3.6-moe:35b-a3b --ctx-len 262144. then, in another terminal window, I run: codex --profile flm
Codex loads, I send in a prompt, the model loads, and I can see the output of the server.
But, I don't actually get a response in Codex, but a warning and an error message:
⚠ Model metadata for `qwen3.6-moe:35b-a3b` not found. Defaulting to fallback metadata; this can degrade performance and cause issues.
■ unexpected status 404 Not Found: {"error":"Not Found"}, url: http://localhost:52625/v1/responses
So, can anyone tell me what am I doing wrong?
(edit: codex CLI no longer supports 'chat' and 'completions', I have to use 'responses', and I see in the documentation that 'responses' endpoint is not supported - so probably that's it)
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reproducing the Windows setup with flm serve qwen3.6-moe:35b-a3b --ctx-len 262144 and the supplied .codex\config.toml. Check the documented API endpoints against Codex's request to /v1/responses; done means the supported integration path is identified and either works or the incompatibility is clearly documented.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- powershell
- Domain
- api, backend-api-design
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100