Runtime treats output-free provider finish frames as successful completion
- Dominant language
- TypeScript
- Stars
- 5.4k
- Forks
- 502
- Avg merge
- 1d 2h
- Merged PRs (30d)
- 715
Description
### What happened
After a tool call has completed, a provider can end the next physical model request with all of the following:
- raw finish reason: `stop`
- no text output
- no reasoning/thinking output
- no tool call or continuation metadata
- unavailable usage fields
Maka currently settles this response as `completed` and maps it to `end_turn`. The task therefore appears to finish successfully even though the model produced no answer and the requested work is still incomplete.
I observed this intermittently with `openrouter/stealth/ox-alpha` after tool execution. The Desktop returns to the idle state without displaying an error. For a Goal-driven task, this can also cause the unfinished turn to enter completion/evaluation handling prematurely.
A related output-free finish whose raw reason is exactly `network_error` is also treated as a normal completion instead of entering the existing network retry path.
Expected behavior:
- An output-free, unmetered `stop` should not be accepted as successful completion.
- Maka should perform a bounded retry.
- If the retry is exhausted, the turn should fail explicitly with `provider_unavailable`.
- A raw `network_error` should use the existing network retry policy.
- Retrying must not execute an already-settled tool again, consume an additional logical step, create a fabricated usage checkpoint, or persist an empty assistant step.
- Legitimate responses such as text with unavailable usage or an empty response with authoritative zero usage must remain valid.
简体中文
工具调用完成后,供应商的下一次物理模型请求可能只返回以下结束信息:
- 原始结束原因是 `stop`
- 没有文本输出
- 没有 reasoning/thinking 输出
- 没有工具调用或 continuation metadata
- usage 字段不可用
Maka 当前会把这种响应判定为 `completed`,随后映射成 `end_turn`。因此,即使模型没有返回任何回答、任务也尚未完成,界面仍会表现为任务已成功结束。
该问题在 `openrouter/stealth/ox-alpha` 执行工具后间歇性出现。Desktop 会直接恢复空闲状态,并且不显示错误。对于启用了 Goal 的任务,未完成的 Turn 也可能提前进入完成或评估流程。
另外,原始结束原因精确为 `network_error` 且没有任何输出的响应,也会被当作正常完成,而没有进入现有网络重试流程。
预期行为:
- 没有输出且没有有效 usage 的 `stop` 不应被判定为成功。
- Maka 应进行有限次数的重试。
- 重试耗尽后,应明确返回 `provider_unavailable`。
- 原始 `network_error` 应进入现有网络重试流程。
- 重试时不能重复执行已经完成的工具、消耗额外逻辑步骤、生成虚假的 usage checkpoint,或保存空的 Assistant step。
- 有文本但缺少 usage、具有权威零 usage 等合法响应必须保持原有行为。
### How to reproduce
Manual reproduction is intermittent because the trigger depends on the provider response.
1. Configure an OpenRouter connection and select `openrouter/stealth/ox-alpha`.
2. Start a multi-step task that requires one or more tool calls.
3. Allow the first model request and its tool call to complete.
4. Have the provider return only an output-free `stop` finish frame for the following model request.
5. Observe that Maka emits a successful completion and the Desktop becomes idle even though no final answer was produced.
A deterministic Runtime-level reproduction can use this stream sequence:
1. First model stream: one tool call followed by a valid `tool-calls` finish.
2. Execute and persist the tool result exactly once.
3. Second model stream: only `finishReason.raw = "stop"`, with no content and all usage fields unavailable.
4. Current `main` settles the turn as `completed` / `end_turn` instead of retrying.
The related network variant can be reproduced by replacing the second finish reason with the exact raw value `network_error`.
简体中文
由于触发条件取决于供应商实际返回的响应,因此手动复现具有间歇性。
1. 配置 OpenRouter 连接并选择 `openrouter/stealth/ox-alpha`。
2. 启动一个需要执行一个或多个工具的多步骤任务。
3. 等待第一次模型请求和工具调用完成。
4. 供应商在下一次模型请求中只返回一个没有任何输出的 `stop` 结束帧。
5. 可以看到 Maka 将任务判定为成功,Desktop 恢复空闲,但没有生成最终回答。
可以通过以下模型流稳定复现 Runtime 行为:
1. 第一次模型流返回一个工具调用,并以有效的 `tool-calls` 结束。
2. 工具结果只执行和持久化一次。
3. 第二次模型流只返回 `finishReason.raw = "stop"`,没有任何内容,并且所有 usage 字段均不可用。
4. 当前 `main` 会把 Turn 判定为 `completed` / `end_turn`,而不是进行重试。
将第二次结束原因替换为精确的原始值 `network_error`,可以复现相关的网络错误分支。
### Environment
- Maka version observed: `0.2.0` development build (`04836d3`)
- Confirmed affected revision: current `main` at `902d785`
- OS: macOS / Darwin `25.5.0` (`arm64`)
- Surface: Desktop / Runtime
- Runtime Node.js: 24.x
- Provider: OpenRouter
- Model: `openrouter/stealth/ox-alpha`
简体中文
- 发现问题的 Maka 版本:`0.2.0` 开发版(`04836d3`)
- 已确认仍受影响的版本:当前 `main`,提交 `902d785`
- 操作系统:macOS / Darwin `25.5.0`(`arm64`)
- 影响范围:Desktop / Runtime
- Runtime Node.js:24.x
- 供应商:OpenRouter
- 模型:`openrouter/stealth/ox-alpha`
### Logs, screenshots, or additional context
Observed terminal model response:
- finish reason: `stop`
- text: none
- reasoning: none
- tool calls: none
- usage: unavailable
- current Maka outcome: `completed` → `end_turn`
The provider request itself finishes without throwing an exception, so the existing exception-based retry path is not activated.
Impact:
- unfinished work is silently treated as completed;
- the UI provides no actionable error;
- long-running or Goal-driven tasks can stop before producing their requested result;
- no data corruption or security impact has been observed.
The first “继续” message at 13:57 produced no assistant response and the turn silently returned to idle. Sending the same message again at 14:05 resumed execution normally.
第一次在 13:57 发送“继续”后,没有产生任何 Assistant 回复,Turn 静默恢复为空闲状态。14:05 再次发送相同消息后,任务才恢复正常执行。
简体中文
观察到的最终模型响应:
- 结束原因:`stop`
- 文本:无
- reasoning:无
- 工具调用:无
- usage:不可用
- Maka 当前结果:`completed` → `end_turn`
供应商请求本身没有抛出异常,因此现有的异常重试流程不会被触发。
影响:
- 尚未完成的工作会被静默判定为成功;
- 界面不会提供可操作的错误提示;
- 长时间运行或启用了 Goal 的任务可能在产生预期结果前停止;
- 目前没有发现数据损坏或安全影响。
Contributor guide
Research direction
Start with the Runtime-level stream sequence described in the issue and trace how output-free finish frames enter the existing completion and network-retry paths. Add deterministic coverage for the tool-call-then-empty-stop and raw network_error cases, verifying bounded retries, explicit provider_unavailable failure, and no duplicate tool execution, logical step, usage checkpoint, or empty assistant step.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- electron, node.js, typescript
- Domain
- ai, backend, desktop
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100