NVIDIA / NVIDIA/Personal-AI-Router

[Bug]: Action HTTP response-header timeout is selected by a hardcoded engine name, so only Ollama gets the long client

オープン
#25 コメント 1 件 リアクション 1 件 担当者 1 名 GitHub で見る

まだ誰も着手していません。

主要言語
Go
スター
1.4k
フォーク
250
平均マージ
23時間 27分
マージ済み PR(30日)
1

説明

PAIR version or commit

0.91.7 (nvpair-engine-manager 0.17.4, linux/arm64). The code paths below are also present on main.

Affected component

Go services — nvpair-engine-manager

Environment

NVIDIA DGX Spark (GB10, aarch64), Ubuntu 24.04.4 LTS, driver 580.173.02, 10-node PAIR cluster. Reproduced against stock binaries with no local modifications.

Steps to reproduce
  1. Configure an engine other than Ollama.
  2. Issue an engine:action whose HTTP call needs more than 30 seconds to return response headers. The realistic case is a chat action against a large model with a long prompt, where prefill alone can exceed 30s before the first header is written.
  3. Observe the action fail with a transport timeout.
Expected behavior

The per-action timeout should be a property of the action or the manifest rather than of the engine's name, so an engine author can declare that a given action may be slow.

Actual behavior

executor.go defines two clients:

engineResponseHeaderTimeout     = 30 * time.Second
ollamaLoadResponseHeaderTimeout = 10 * time.Minute

and actions.go chooses between them with a literal engine-name comparison:

client := e.client
if engine == "ollama" && action == "run_model" && e.ollamaLoadClient != nil {
    client = e.ollamaLoadClient
}

Every other engine therefore gets 30 seconds for every action, and no manifest field can change it.

By code inspection this is not limited to third-party engines. The shipped lmstudio.json declares a chat action (POST /v1/chat/completions) that uses the 30s client, and the vLLM manifest proposed in #9 declares a chat action as well, so a first-class vLLM engine would inherit the same ceiling.

The failure also presents badly: net/http: timeout awaiting response headers reads as a broken or hung engine rather than a client-side limit.

Sanitized logs or screenshots
action "run_model": Post "http://127.0.0.1:8800/v1/chat/completions":
net/http: timeout awaiting response headers      [30s]

Scope of what I actually observed, to be precise: I hit this with an engine registered through a user manifest in <appdir>/engines/, driving engine-manager over its stdio JSON-RPC. It reproduced consistently while the model was cold and stopped once I pre-warmed the model during load, which is what isolated it to the response-header timeout rather than to the engine itself. I have not reproduced a 30s chat failure on LM Studio directly; that exposure is read from its manifest and the selection logic above.

Suggested fix

An optional per-action timeout_s in the manifest, defaulting to the current 30s, would cover this and would let the Ollama exception move into ollama.json instead of living in Go — removing the engine-name comparison entirely.

コントリビューションガイド

コントリビューションガイドを開く

はじめの一歩

  1. issue を最後まで読み、次にプロジェクトのコントリビューションガイドを読みます。
  2. 着手することを issue にコメントします — 二人が同じ作業をするのを防げます。
  3. リポジトリをフォークし、ブランチを切って変更します。
  4. issue 番号を参照したプルリクエストを送ります。

評価

この issue はまだ評価されていません。

新しい issue をメールで受け取る

初心者向けの GitHub issue を短くまとめたダイジェスト。