Fail fast when fixed request overhead exceeds the model's usable context window

Fail fast when fixed request overhead exceeds the model's usable context window

Summary

Follow-up to #48827 (auto-compaction loop guard) and #48844 (chunked compaction + tail escalation, PR #48845). With both in place, a third failure mode remains in this family: when a model's usable window (limit.context − maxOutputTokens, session/overflow.ts) is smaller than the request's fixed portion — system prompt + tool/MCP schemas + project instructions — every request overflows regardless of conversation content. Compaction can only shrink conversation content, never the fixed portion, so the outcome is knowable before the first attempt — yet the session loop still runs the full ladder: auto-compact → resume → overflow → escalate retained tail (full → half → none) → repeat, until the loop guard stops the session at attempt 3.

Evidence (live, desktop v1.18.30 with #48845 applied)

Project fixed floor — provider-reported tokens on the first request of fresh sessions, before any conversation content: 34,474–34,771 tokens.

cerebras/qwen-3.8-27b: limit.context = 65,536, limit.output = 32,768 → usable = 65,536 − min(32,768, 32,000) = 33,536.

Floor (34.5k) > usable (33.5k) — even an empty session overflows on its first turn in this project.

Observed sequence on a ~106k-token session after switching to this model:

1. Auto-compaction succeeded: the 105,817-token head was summarized in a single request (1M-window compaction model deepseek/deepseek-v4-flash), producing a 3,254-token summary — the #48845 path working as designed.
2. The resumed request registered 54,499 tokens > 33,536 → second auto-compaction (input 11,516 — only the new remainder; tail escalation engaged).
3. The resumed request still overflowed; the user cancelled before the guard's third attempt.

Every component behaved as designed, but ~117k compaction input tokens and ~40s of "compacting" were spent reaching an outcome that was determined before attempt 1.

Proposed behavior

Detect the impossible fit and fail fast with a targeted error, e.g.:

This model's usable context window (33,536 tokens: 65,536 context − 32,000 output reservation) is smaller than this project's baseline request size (~34,500 tokens of system prompt, tool definitions, and project instructions). Compaction cannot help — choose a model with a larger context window, or reduce enabled tools/MCP servers and instruction files.

Detection sketch: at the auto-compaction trigger in session/prompt.ts (or inside SessionCompaction.select()), estimate the fixed portion — Token.estimate over system prompt + tool schemas + instruction files — and compare it against usable(model). If the floor exceeds usable (or usable minus a minimal summary allowance), skip compaction entirely and raise the error. The same check could later power a proactive warning when the user switches to an undersized model.

Related

• #48827 — auto-compaction loop guard
• #48844 — chunked summarization + tail escalation (PR #48845)

Environment

• OpenCode desktop v1.18.30 (macOS arm64), session model cerebras/qwen-3.8-27b, compaction model deepseek/deepseek-v4-flash, project with MCP servers and instruction files enabled.