Claude 5-generation models (adaptive_thinking: "required") never return readable reasoning text - only opaque envelope, despite billed reasoningTokens
- 主要语言
- Java
- 星标
- 10.5k
- 派生
- 1.5k
- 平均合并
- 1 天 14 小时
- 30 天内合并 PR
- 129
描述
## Summary
`@github/copilot-sdk`-driven sessions on Claude 5-generation models (`claude-sonnet-5`, `claude-opus-5`, `claude-opus-4.8`) never receive readable reasoning/thinking text (`assistant.reasoning.content` and `assistant.message.reasoningText` are always empty, `assistant.reasoning_delta` never fires), even though the model genuinely performs extended thinking (`assistant.usage.reasoningTokens` is real and substantial) and the request is not being blocked (`contentFilterTriggered: false`). Only the opaque/encrypted envelope (`reasoningOpaque` on `assistant.message`, byte-identical to `reasoningId` on `assistant.reasoning`) ever comes through.
The SDK exposes no configuration surface to influence this at all.
## Environment
- `@github/copilot-sdk`: 1.0.9
- Copilot CLI (bundled runtime): 1.0.80, protocol 1.0.77
- Models affected: `claude-sonnet-5`, `claude-opus-5`, `claude-opus-4.8` (all report `capabilities.supports.adaptive_thinking: "required"` via `listModels()`)
- Models NOT affected: `claude-sonnet-4.6` (`adaptive_thinking: "optional"`) - not directly re-tested for this specific symptom, but flagged as a likely-safe comparison point; `claude-haiku-4.5` (`adaptive_thinking: "unsupported"`, and also doesn't support `reasoningEffort` at all)
- GPT-family models (e.g. `gpt-5.6-luna`) are unaffected - reasoning streams and finalizes with full readable text correctly, confirmed live.
## What we found, and how we verified it (no reverse engineering involved)
1. **`listModels()` reports `adaptive_thinking: "required"` for the affected models.** Called `client.listModels()` directly and inspected the raw `capabilities.supports` object:
```json
{
"id": "claude-sonnet-5",
"capabilities": {
"supports": {
"adaptive_thinking": "required",
"max_thinking_budget": 32000,
"min_thinking_budget": 1024,
...
}
}
}
```
Per the SDK's own doc comment on `AdaptiveThinkingSupport` (`generated/rpc.d.ts`): `"required"` means *"The model only accepts adaptive thinking and rejects `thinking.type='enabled'` with HTTP 400."*
2. **There is no SDK-level way to request `thinking: {type: 'adaptive'}` (or any thinking mode) at all.** A full-text search of the entire consumer-facing `dist/types.d.ts` for "thinking" or "adaptive" returns zero matches. `SessionConfigBase.reasoningEffort` and `SessionConfigBase.reasoningSummary` are the only reasoning-related session options exposed, and neither is documented as, or appears to actually be, a substitute for Anthropic's native `thinking` request parameter.
3. **The observed data is fully consistent with "adaptive thinking runs, but the runtime doesn't surface it as text for this negotiation path" rather than any kind of safety filtering:**
- `assistant.usage.reasoningTokens` is real and often substantial (26-495 tokens observed across many calls) - the model is genuinely spending tokens on extended thinking.
- `assistant.usage.contentFilterTriggered` is `false` on every single affected call.
- `assistant.usage.apiEndpoint` is `/v1/messages` (the Anthropic-native endpoint) on every call, both the ones that come back with readable text and the ones that don't - so it isn't a difference in which endpoint is used.
- `assistant.message.reasoningOpaque` and `assistant.reasoning.reasoningId` are byte-for-byte identical for the same reasoning block - the same encrypted envelope is surfaced on two separate event fields, with no plaintext on either.
- `assistant.reasoning_delta` never fires at all for these calls (confirmed via direct count, zero rows).
4. **We ruled out every other explanation we could think of via direct, controlled tests before landing here** (documented in full in our own investigation notes, happy to share if useful): `streaming` session option, ephemeral-event/resume-replay behavior, sub-agent involvement, ephemeral vs. persisted event handling, enterprise/managed-settings policy (confirmed `session.managed_settings_resolved`/`enforced` never fire for any of our sessions), reasoning-token-count thresholds, and repository content/context (a controlled A/B test running an unrelated generic prompt inside the exact same repository that reliably showed 17/17 redacted results for real work came back fully clean, ruling out repo-specific policy or content-classification as the cause).
## Why we believe this is specifically an SDK/runtime gap, not intended Anthropic behavior
The interactive `copilot` CLI TUI and the VS Code Copilot Chat extension both reliably show full reasoning text for the exact same account, models, and repository/content. VS Code's own (open source) implementation constructs the Anthropic `/v1/messages` request directly, explicitly setting `thinking: { type: 'adaptive' }` (or `{ type: 'enabled', budget_tokens }` for non-required models) plus the `interleaved-thinking-2025-05-14` beta header, and accumulates the raw `thinking`/`signature` deltas itself - it does not depend on any server-side summarization step. This strongly suggests the underlying model/API absolutely can and does return readable thinking text for these models when the request negotiates adaptive thinking correctly; the gap appears to be specific to how `@github/copilot-sdk`-driven sessions (and/or the CLI runtime backing them) negotiate or expose this for `adaptive_thinking: "required"` models.
## Ask
1. Could the SDK expose a session-level option to control Anthropic's `thinking` parameter directly (matching what VS Code's own client already sends), so consumers aren't dependent on an internal summarization step that doesn't appear to work for `adaptive_thinking: "required"` models?
2. Alternatively/additionally, could `assistant.reasoning`/`assistant.message` reliably carry readable text for these models the way they already do for GPT-family reasoning models?
Happy to provide full raw event logs, or the request/response correlation IDs we've already captured on affected calls (`interactionId`, `serviceRequestId` / `x-copilot-service-request-id`, `providerCallId` / `x-github-request-id`, `apiCallId`) if useful for looking up server-side records.
贡献指南
调研方向
从 generated/rpc.d.ts 和 dist/types.d.ts 开始,检查公开的 reasoning 和 adaptive-thinking 类型,然后将会话请求路径与 issue 中描述的 VS Code 实现进行比较。验证 listModels() 如何报告受影响的模型,以及 assistant reasoning 事件如何公开。完成的标准是:能够协商所需的 adaptive-thinking 模型,并且能够可靠地公开可读的 reasoning,同时测试覆盖事件输出。
由索引模型根据 Issue 内容生成。
评估
- 技术栈
- github, vscode
- 领域
- api, devtools
- Issue 类型
- 缺陷
- 难度
- 4/5
- 预计耗时
- 3-5 天
- 活跃度
- 活跃
- 描述清晰度
- 基本清楚
- 新手友好度
- 42/100