apache / apache/maka

bug(runtime): a single attached image trips context_budget_exhausted on the first turn (image bytes counted as raw serialized chars)

Open
#4,290 1 comment 0 reactions 1 assignee Claimed by @liuxiaocs7 View on GitHub
Dominant language
TypeScript
Stars
5.4k
Forks
502
Avg merge
1d 2h
Merged PRs (30d)
715

Description

### What happened

In Maka Desktop, sending a **first-turn** message that carries a single **user-uploaded image** (with a custom/relay model selected) fails immediately with a context-limit error, even though nothing has been sent to the provider yet:

```
Surface: toast
Title: 任务出错
Description: 上下文已达上限,这个任务无法继续。换模型或开启新任务。
Reason: context_budget_exhausted
Details: {"contextBudgetExhaustedDetail":"no_safe_completed_span"}
```

The runtime execution record shows the turn dying in `263ms` with `Steps (0)` — the request never reaches the model. This is a **false positive**: the runtime's own pre-flight capacity estimate counts the attached image at the size of its transported bytes, which for one screenshot is enough to blow past the model's context window on its own.

### Impact

Reproducible whenever **both** conditions hold:

1. The selected model has **no known context window** (a custom/relay model; the AI SDK also logs `The model "" is unknown`). This makes `resolveContextBudgetCapacity` return `source: 'policy_fallback'` with `tokens = 32_000 + 16_384 = 48_384` (`packages/runtime/src/context-budget-policy.ts`, confirmed by `context-budget-mid-turn-policy.test.ts`).
2. The message carries at least one image attachment.

Text-only first turns are fine; known-window models are fine on step 0 (the step-0 verdict is skipped for them).

### Root cause

Two payload estimators disagree on how to count images:

- The RuntimeEvent/ModelMessage **history** estimators (`estimateRuntimeEventChars` in `context-budget-helpers.ts`, `estimatePartChars` in `context-budget.ts`) deliberately **skip** image binary — they count only `text`/`toolName`/`toolCallId`/`input`/`output`. (This is the #3372 direction: images count as zero.)
- The **capacity verdict** estimator `midTurnRequestPayloadChars` (`packages/runtime/src/ai-sdk-compaction.ts`) uses:

```ts
systemPromptChars + JSON.stringify(messages).length + toolSchemaChars
```

`JSON.stringify(messages)` serializes the inline image payload at its raw transported size. A current-turn user image is materialized as `{ type: 'file', data: { type: 'data', data: }, mediaType }` (`ai-sdk-backend.ts` `appendImageParts`); `JSON.stringify(Uint8Array)` expands each byte to a `"":` entry (~10 chars/byte), so a ~150 KB image becomes >1 MB of JSON. Tool-result images are inline base64 strings and are counted at full length. Either way `chars / 4` reads as hundreds of thousands of "tokens" when the provider will bill the rendered image at a bounded cost (~1.6K tokens for a vision model). The code even acknowledges this in `context-diagnostics.ts`: *"`bytes / 4` is a rule of thumb over serialized JSON — wrong in a direction nobody here can correct for, badly so for an attachment's base64"* — but that caveat only guards the diagnostics display, not the capacity verdict.

Chain:

1. Unknown model ⇒ `policy_fallback` capacity (`48_384` tokens).
2. `policy_fallback` makes `buildMidTurnFinalRequestVerdict` run its capacity verdict on **step 0** (`ai-sdk-compaction.ts`: `options.stepNumber >= 1 || state.capacity.source === 'policy_fallback'`), i.e. before the first request is sent.
3. `midTurnRequestPayloadChars` over-counts the image by its serialized bytes ⇒ `estimate ≫ capacity.tokens`.
4. It's the first turn — the current user message is the pinned head anchor, so there is no completed prefix to fold ⇒ `selectSafeCompactionPrefix` returns `no_safe_completed_span`.
5. The verdict aborts the turn with `context_budget_exhausted` / `no_safe_completed_span` before any provider round-trip.

### How to reproduce

1. Configure a custom/relay model whose context window is unknown to Maka (no declared `contextWindow`, no metadata).
2. Start a fresh conversation.
3. Attach one image and send a message (e.g. "看看这张图包含什么内容").
4. Observe: an immediate `context_budget_exhausted` toast; the runtime record shows `Steps (0)`, ~200–300 ms, no provider request.

### Proposed fix

Make `midTurnRequestPayloadChars` image-aware: keep every byte of structural overhead but count each inline image/file **binary** payload at a bounded per-image estimate (≈ the vision per-image ceiling) instead of its raw serialized byte size. This avoids both the over-count (this bug) and the #3372 under-count (image = 0). A follow-up hardening is to not let `policy_fallback` emit a terminal `context_budget_exhausted` for a first, unshrinkable user message — let the provider's real token count decide via the existing reactive recovery (#3412) — and/or to let relay models declare a `contextWindow`.

### Related

- #3372 / #3412 — the *opposite* direction (hydrated image payloads counted as **zero**, reactive omission after a provider overflow). Does not cover this proactive, pre-send over-count of a current-turn user image.
- #4288 — a separate, already-filed bug present in the same diagnostic report's logs (branch/fork of a conversation with an attachment fails with `persistence_failed`). Independent of this context-budget issue.

### Environment

- Maka: 0.2.0 (dev @ `66e6f4e`)
- Electron 43.4.1 · Chrome 150 · Node 24.18.1
- OS: darwin 24.6.0 (arm64)
- Locale: zh-CN
- Runtime Host: Protocol v0 · compatibility 76
- Model: a custom relay model with no context window known to Maka (unknown to the AI SDK)

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.