apache / apache/maka

Runtime treats output-free provider finish frames as successful completion

Open
#3,772 1 comment 0 reactions 0 assignees View on GitHub
bug help wanted
Dominant language
TypeScript
Stars
5.4k
Forks
502
Avg merge
1d 2h
Merged PRs (30d)
715

Description

### What happened

After a tool call has completed, a provider can end the next physical model request with all of the following:

- raw finish reason: `stop`
- no text output
- no reasoning/thinking output
- no tool call or continuation metadata
- unavailable usage fields

Maka currently settles this response as `completed` and maps it to `end_turn`. The task therefore appears to finish successfully even though the model produced no answer and the requested work is still incomplete.

I observed this intermittently with `openrouter/stealth/ox-alpha` after tool execution. The Desktop returns to the idle state without displaying an error. For a Goal-driven task, this can also cause the unfinished turn to enter completion/evaluation handling prematurely.

A related output-free finish whose raw reason is exactly `network_error` is also treated as a normal completion instead of entering the existing network retry path.

Expected behavior:

- An output-free, unmetered `stop` should not be accepted as successful completion.
- Maka should perform a bounded retry.
- If the retry is exhausted, the turn should fail explicitly with `provider_unavailable`.
- A raw `network_error` should use the existing network retry policy.
- Retrying must not execute an already-settled tool again, consume an additional logical step, create a fabricated usage checkpoint, or persist an empty assistant step.
- Legitimate responses such as text with unavailable usage or an empty response with authoritative zero usage must remain valid.

简体中文

工具调用完成后,供应商的下一次物理模型请求可能只返回以下结束信息:

- 原始结束原因是 `stop`
- 没有文本输出
- 没有 reasoning/thinking 输出
- 没有工具调用或 continuation metadata
- usage 字段不可用

Maka 当前会把这种响应判定为 `completed`,随后映射成 `end_turn`。因此,即使模型没有返回任何回答、任务也尚未完成,界面仍会表现为任务已成功结束。

该问题在 `openrouter/stealth/ox-alpha` 执行工具后间歇性出现。Desktop 会直接恢复空闲状态,并且不显示错误。对于启用了 Goal 的任务,未完成的 Turn 也可能提前进入完成或评估流程。

另外,原始结束原因精确为 `network_error` 且没有任何输出的响应,也会被当作正常完成,而没有进入现有网络重试流程。

预期行为:

- 没有输出且没有有效 usage 的 `stop` 不应被判定为成功。
- Maka 应进行有限次数的重试。
- 重试耗尽后,应明确返回 `provider_unavailable`。
- 原始 `network_error` 应进入现有网络重试流程。
- 重试时不能重复执行已经完成的工具、消耗额外逻辑步骤、生成虚假的 usage checkpoint,或保存空的 Assistant step。
- 有文本但缺少 usage、具有权威零 usage 等合法响应必须保持原有行为。

### How to reproduce

Manual reproduction is intermittent because the trigger depends on the provider response.

1. Configure an OpenRouter connection and select `openrouter/stealth/ox-alpha`.
2. Start a multi-step task that requires one or more tool calls.
3. Allow the first model request and its tool call to complete.
4. Have the provider return only an output-free `stop` finish frame for the following model request.
5. Observe that Maka emits a successful completion and the Desktop becomes idle even though no final answer was produced.

A deterministic Runtime-level reproduction can use this stream sequence:

1. First model stream: one tool call followed by a valid `tool-calls` finish.
2. Execute and persist the tool result exactly once.
3. Second model stream: only `finishReason.raw = "stop"`, with no content and all usage fields unavailable.
4. Current `main` settles the turn as `completed` / `end_turn` instead of retrying.

The related network variant can be reproduced by replacing the second finish reason with the exact raw value `network_error`.

简体中文

由于触发条件取决于供应商实际返回的响应,因此手动复现具有间歇性。

1. 配置 OpenRouter 连接并选择 `openrouter/stealth/ox-alpha`。
2. 启动一个需要执行一个或多个工具的多步骤任务。
3. 等待第一次模型请求和工具调用完成。
4. 供应商在下一次模型请求中只返回一个没有任何输出的 `stop` 结束帧。
5. 可以看到 Maka 将任务判定为成功,Desktop 恢复空闲,但没有生成最终回答。

可以通过以下模型流稳定复现 Runtime 行为:

1. 第一次模型流返回一个工具调用,并以有效的 `tool-calls` 结束。
2. 工具结果只执行和持久化一次。
3. 第二次模型流只返回 `finishReason.raw = "stop"`,没有任何内容,并且所有 usage 字段均不可用。
4. 当前 `main` 会把 Turn 判定为 `completed` / `end_turn`,而不是进行重试。

将第二次结束原因替换为精确的原始值 `network_error`,可以复现相关的网络错误分支。

### Environment

- Maka version observed: `0.2.0` development build (`04836d3`)
- Confirmed affected revision: current `main` at `902d785`
- OS: macOS / Darwin `25.5.0` (`arm64`)
- Surface: Desktop / Runtime
- Runtime Node.js: 24.x
- Provider: OpenRouter
- Model: `openrouter/stealth/ox-alpha`

简体中文

- 发现问题的 Maka 版本:`0.2.0` 开发版(`04836d3`)
- 已确认仍受影响的版本:当前 `main`,提交 `902d785`
- 操作系统:macOS / Darwin `25.5.0`(`arm64`)
- 影响范围:Desktop / Runtime
- Runtime Node.js:24.x
- 供应商:OpenRouter
- 模型:`openrouter/stealth/ox-alpha`

### Logs, screenshots, or additional context

Observed terminal model response:

- finish reason: `stop`
- text: none
- reasoning: none
- tool calls: none
- usage: unavailable
- current Maka outcome: `completed` → `end_turn`

The provider request itself finishes without throwing an exception, so the existing exception-based retry path is not activated.

Impact:

- unfinished work is silently treated as completed;
- the UI provides no actionable error;
- long-running or Goal-driven tasks can stop before producing their requested result;
- no data corruption or security impact has been observed.

Image

The first “继续” message at 13:57 produced no assistant response and the turn silently returned to idle. Sending the same message again at 14:05 resumed execution normally.

第一次在 13:57 发送“继续”后,没有产生任何 Assistant 回复,Turn 静默恢复为空闲状态。14:05 再次发送相同消息后,任务才恢复正常执行。

简体中文

观察到的最终模型响应:

- 结束原因:`stop`
- 文本:无
- reasoning:无
- 工具调用:无
- usage:不可用
- Maka 当前结果:`completed` → `end_turn`

供应商请求本身没有抛出异常,因此现有的异常重试流程不会被触发。

影响:

- 尚未完成的工作会被静默判定为成功;
- 界面不会提供可操作的错误提示;
- 长时间运行或启用了 Goal 的任务可能在产生预期结果前停止;
- 目前没有发现数据损坏或安全影响。

Contributor guide

Open the contributing guide

Research direction

Start with the Runtime-level stream sequence described in the issue and trace how output-free finish frames enter the existing completion and network-retry paths. Add deterministic coverage for the tool-call-then-empty-stop and raw network_error cases, verifying bounded retries, explicit provider_unavailable failure, and no duplicate tool execution, logical step, usage checkpoint, or empty assistant step.

Written by the indexing model from the issue text.

Assessment

Tech stack
electron, node.js, typescript
Domain
ai, backend, desktop
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.