bug(runtime): a single attached image trips context_budget_exhausted on the first turn (image bytes counted as raw serialized chars)
- Dominant language
- TypeScript
- Stars
- 5.4k
- Forks
- 502
- Avg merge
- 1d 2h
- Merged PRs (30d)
- 715
Description
### What happened
In Maka Desktop, sending a **first-turn** message that carries a single **user-uploaded image** (with a custom/relay model selected) fails immediately with a context-limit error, even though nothing has been sent to the provider yet:
```
Surface: toast
Title: 任务出错
Description: 上下文已达上限,这个任务无法继续。换模型或开启新任务。
Reason: context_budget_exhausted
Details: {"contextBudgetExhaustedDetail":"no_safe_completed_span"}
```
The runtime execution record shows the turn dying in `263ms` with `Steps (0)` — the request never reaches the model. This is a **false positive**: the runtime's own pre-flight capacity estimate counts the attached image at the size of its transported bytes, which for one screenshot is enough to blow past the model's context window on its own.
### Impact
Reproducible whenever **both** conditions hold:
1. The selected model has **no known context window** (a custom/relay model; the AI SDK also logs `The model "" is unknown`). This makes `resolveContextBudgetCapacity` return `source: 'policy_fallback'` with `tokens = 32_000 + 16_384 = 48_384` (`packages/runtime/src/context-budget-policy.ts`, confirmed by `context-budget-mid-turn-policy.test.ts`).
2. The message carries at least one image attachment.
Text-only first turns are fine; known-window models are fine on step 0 (the step-0 verdict is skipped for them).
### Root cause
Two payload estimators disagree on how to count images:
- The RuntimeEvent/ModelMessage **history** estimators (`estimateRuntimeEventChars` in `context-budget-helpers.ts`, `estimatePartChars` in `context-budget.ts`) deliberately **skip** image binary — they count only `text`/`toolName`/`toolCallId`/`input`/`output`. (This is the #3372 direction: images count as zero.)
- The **capacity verdict** estimator `midTurnRequestPayloadChars` (`packages/runtime/src/ai-sdk-compaction.ts`) uses:
```ts
systemPromptChars + JSON.stringify(messages).length + toolSchemaChars
```
`JSON.stringify(messages)` serializes the inline image payload at its raw transported size. A current-turn user image is materialized as `{ type: 'file', data: { type: 'data', data: }, mediaType }` (`ai-sdk-backend.ts` `appendImageParts`); `JSON.stringify(Uint8Array)` expands each byte to a `"":` entry (~10 chars/byte), so a ~150 KB image becomes >1 MB of JSON. Tool-result images are inline base64 strings and are counted at full length. Either way `chars / 4` reads as hundreds of thousands of "tokens" when the provider will bill the rendered image at a bounded cost (~1.6K tokens for a vision model). The code even acknowledges this in `context-diagnostics.ts`: *"`bytes / 4` is a rule of thumb over serialized JSON — wrong in a direction nobody here can correct for, badly so for an attachment's base64"* — but that caveat only guards the diagnostics display, not the capacity verdict.
Chain:
1. Unknown model ⇒ `policy_fallback` capacity (`48_384` tokens).
2. `policy_fallback` makes `buildMidTurnFinalRequestVerdict` run its capacity verdict on **step 0** (`ai-sdk-compaction.ts`: `options.stepNumber >= 1 || state.capacity.source === 'policy_fallback'`), i.e. before the first request is sent.
3. `midTurnRequestPayloadChars` over-counts the image by its serialized bytes ⇒ `estimate ≫ capacity.tokens`.
4. It's the first turn — the current user message is the pinned head anchor, so there is no completed prefix to fold ⇒ `selectSafeCompactionPrefix` returns `no_safe_completed_span`.
5. The verdict aborts the turn with `context_budget_exhausted` / `no_safe_completed_span` before any provider round-trip.
### How to reproduce
1. Configure a custom/relay model whose context window is unknown to Maka (no declared `contextWindow`, no metadata).
2. Start a fresh conversation.
3. Attach one image and send a message (e.g. "看看这张图包含什么内容").
4. Observe: an immediate `context_budget_exhausted` toast; the runtime record shows `Steps (0)`, ~200–300 ms, no provider request.
### Proposed fix
Make `midTurnRequestPayloadChars` image-aware: keep every byte of structural overhead but count each inline image/file **binary** payload at a bounded per-image estimate (≈ the vision per-image ceiling) instead of its raw serialized byte size. This avoids both the over-count (this bug) and the #3372 under-count (image = 0). A follow-up hardening is to not let `policy_fallback` emit a terminal `context_budget_exhausted` for a first, unshrinkable user message — let the provider's real token count decide via the existing reactive recovery (#3412) — and/or to let relay models declare a `contextWindow`.
### Related
- #3372 / #3412 — the *opposite* direction (hydrated image payloads counted as **zero**, reactive omission after a provider overflow). Does not cover this proactive, pre-send over-count of a current-turn user image.
- #4288 — a separate, already-filed bug present in the same diagnostic report's logs (branch/fork of a conversation with an attachment fails with `persistence_failed`). Independent of this context-budget issue.
### Environment
- Maka: 0.2.0 (dev @ `66e6f4e`)
- Electron 43.4.1 · Chrome 150 · Node 24.18.1
- OS: darwin 24.6.0 (arm64)
- Locale: zh-CN
- Runtime Host: Protocol v0 · compatibility 76
- Model: a custom relay model with no context window known to Maka (unknown to the AI SDK)
Contributor guide
Assessment
This issue has not been assessed yet.