NVIDIA / NVIDIA/Personal-AI-Router
[Bug]: Action HTTP response-header timeout is selected by a hardcoded engine name, so only Ollama gets the long client
Nobody has claimed this yet.
- Dominant language
- Go
- Stars
- 1.4k
- Forks
- 250
- Avg merge
- 23h 27m
- Merged PRs (30d)
- 1
Description
PAIR version or commit
0.91.7 (nvpair-engine-manager 0.17.4, linux/arm64). The code paths below are also present on main.
Affected component
Go services — nvpair-engine-manager
Environment
NVIDIA DGX Spark (GB10, aarch64), Ubuntu 24.04.4 LTS, driver 580.173.02, 10-node PAIR cluster. Reproduced against stock binaries with no local modifications.
Steps to reproduce
- Configure an engine other than Ollama.
- Issue an
engine:actionwhose HTTP call needs more than 30 seconds to return response headers. The realistic case is achataction against a large model with a long prompt, where prefill alone can exceed 30s before the first header is written. - Observe the action fail with a transport timeout.
Expected behavior
The per-action timeout should be a property of the action or the manifest rather than of the engine's name, so an engine author can declare that a given action may be slow.
Actual behavior
executor.go defines two clients:
engineResponseHeaderTimeout = 30 * time.Second
ollamaLoadResponseHeaderTimeout = 10 * time.Minute
and actions.go chooses between them with a literal engine-name comparison:
client := e.client
if engine == "ollama" && action == "run_model" && e.ollamaLoadClient != nil {
client = e.ollamaLoadClient
}
Every other engine therefore gets 30 seconds for every action, and no manifest field can change it.
By code inspection this is not limited to third-party engines. The shipped lmstudio.json declares a chat action (POST /v1/chat/completions) that uses the 30s client, and the vLLM manifest proposed in #9 declares a chat action as well, so a first-class vLLM engine would inherit the same ceiling.
The failure also presents badly: net/http: timeout awaiting response headers reads as a broken or hung engine rather than a client-side limit.
Sanitized logs or screenshots
action "run_model": Post "http://127.0.0.1:8800/v1/chat/completions":
net/http: timeout awaiting response headers [30s]
Scope of what I actually observed, to be precise: I hit this with an engine registered through a user manifest in <appdir>/engines/, driving engine-manager over its stdio JSON-RPC. It reproduced consistently while the model was cold and stopped once I pre-warmed the model during load, which is what isolated it to the response-header timeout rather than to the engine itself. I have not reproduced a 30s chat failure on LM Studio directly; that exposure is read from its manifest and the selection logic above.
Suggested fix
An optional per-action timeout_s in the manifest, defaulting to the current 30s, would cover this and would let the Ollama exception move into ollama.json instead of living in Go — removing the engine-name comparison entirely.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.