github / github/copilot-sdk

Claude 5-generation models (adaptive_thinking: "required") never return readable reasoning text - only opaque envelope, despite billed reasoningTokens

Abierto
#2,352 0 comentarios 0 reacciones 0 asignados Ver en GitHub
enhancement
Lenguaje dominante
Java
Estrellas
10.5k
Forks
1.5k
Merge medio
1 d 11 h
PR fusionados (30 d)
128

Descripción

## Summary

`@github/copilot-sdk`-driven sessions on Claude 5-generation models (`claude-sonnet-5`, `claude-opus-5`, `claude-opus-4.8`) never receive readable reasoning/thinking text (`assistant.reasoning.content` and `assistant.message.reasoningText` are always empty, `assistant.reasoning_delta` never fires), even though the model genuinely performs extended thinking (`assistant.usage.reasoningTokens` is real and substantial) and the request is not being blocked (`contentFilterTriggered: false`). Only the opaque/encrypted envelope (`reasoningOpaque` on `assistant.message`, byte-identical to `reasoningId` on `assistant.reasoning`) ever comes through.

The SDK exposes no configuration surface to influence this at all.

## Environment

- `@github/copilot-sdk`: 1.0.9
- Copilot CLI (bundled runtime): 1.0.80, protocol 1.0.77
- Models affected: `claude-sonnet-5`, `claude-opus-5`, `claude-opus-4.8` (all report `capabilities.supports.adaptive_thinking: "required"` via `listModels()`)
- Models NOT affected: `claude-sonnet-4.6` (`adaptive_thinking: "optional"`) - not directly re-tested for this specific symptom, but flagged as a likely-safe comparison point; `claude-haiku-4.5` (`adaptive_thinking: "unsupported"`, and also doesn't support `reasoningEffort` at all)
- GPT-family models (e.g. `gpt-5.6-luna`) are unaffected - reasoning streams and finalizes with full readable text correctly, confirmed live.

## What we found, and how we verified it (no reverse engineering involved)

1. **`listModels()` reports `adaptive_thinking: "required"` for the affected models.** Called `client.listModels()` directly and inspected the raw `capabilities.supports` object:
```json
{
"id": "claude-sonnet-5",
"capabilities": {
"supports": {
"adaptive_thinking": "required",
"max_thinking_budget": 32000,
"min_thinking_budget": 1024,
...
}
}
}
```
Per the SDK's own doc comment on `AdaptiveThinkingSupport` (`generated/rpc.d.ts`): `"required"` means *"The model only accepts adaptive thinking and rejects `thinking.type='enabled'` with HTTP 400."*

2. **There is no SDK-level way to request `thinking: {type: 'adaptive'}` (or any thinking mode) at all.** A full-text search of the entire consumer-facing `dist/types.d.ts` for "thinking" or "adaptive" returns zero matches. `SessionConfigBase.reasoningEffort` and `SessionConfigBase.reasoningSummary` are the only reasoning-related session options exposed, and neither is documented as, or appears to actually be, a substitute for Anthropic's native `thinking` request parameter.

3. **The observed data is fully consistent with "adaptive thinking runs, but the runtime doesn't surface it as text for this negotiation path" rather than any kind of safety filtering:**
- `assistant.usage.reasoningTokens` is real and often substantial (26-495 tokens observed across many calls) - the model is genuinely spending tokens on extended thinking.
- `assistant.usage.contentFilterTriggered` is `false` on every single affected call.
- `assistant.usage.apiEndpoint` is `/v1/messages` (the Anthropic-native endpoint) on every call, both the ones that come back with readable text and the ones that don't - so it isn't a difference in which endpoint is used.
- `assistant.message.reasoningOpaque` and `assistant.reasoning.reasoningId` are byte-for-byte identical for the same reasoning block - the same encrypted envelope is surfaced on two separate event fields, with no plaintext on either.
- `assistant.reasoning_delta` never fires at all for these calls (confirmed via direct count, zero rows).

4. **We ruled out every other explanation we could think of via direct, controlled tests before landing here** (documented in full in our own investigation notes, happy to share if useful): `streaming` session option, ephemeral-event/resume-replay behavior, sub-agent involvement, ephemeral vs. persisted event handling, enterprise/managed-settings policy (confirmed `session.managed_settings_resolved`/`enforced` never fire for any of our sessions), reasoning-token-count thresholds, and repository content/context (a controlled A/B test running an unrelated generic prompt inside the exact same repository that reliably showed 17/17 redacted results for real work came back fully clean, ruling out repo-specific policy or content-classification as the cause).

## Why we believe this is specifically an SDK/runtime gap, not intended Anthropic behavior

The interactive `copilot` CLI TUI and the VS Code Copilot Chat extension both reliably show full reasoning text for the exact same account, models, and repository/content. VS Code's own (open source) implementation constructs the Anthropic `/v1/messages` request directly, explicitly setting `thinking: { type: 'adaptive' }` (or `{ type: 'enabled', budget_tokens }` for non-required models) plus the `interleaved-thinking-2025-05-14` beta header, and accumulates the raw `thinking`/`signature` deltas itself - it does not depend on any server-side summarization step. This strongly suggests the underlying model/API absolutely can and does return readable thinking text for these models when the request negotiates adaptive thinking correctly; the gap appears to be specific to how `@github/copilot-sdk`-driven sessions (and/or the CLI runtime backing them) negotiate or expose this for `adaptive_thinking: "required"` models.

## Ask

1. Could the SDK expose a session-level option to control Anthropic's `thinking` parameter directly (matching what VS Code's own client already sends), so consumers aren't dependent on an internal summarization step that doesn't appear to work for `adaptive_thinking: "required"` models?
2. Alternatively/additionally, could `assistant.reasoning`/`assistant.message` reliably carry readable text for these models the way they already do for GPT-family reasoning models?

Happy to provide full raw event logs, or the request/response correlation IDs we've already captured on affected calls (`interactionId`, `serviceRequestId` / `x-copilot-service-request-id`, `providerCallId` / `x-github-request-id`, `apiCallId`) if useful for looking up server-side records.

Guía de contribución

Abrir la guía de contribución

Línea de trabajo

Start with generated/rpc.d.ts and dist/types.d.ts to review the exposed reasoning and adaptive-thinking types, then compare the session request path with the VS Code implementation described in the issue. Verify how affected models are reported by listModels() and how assistant reasoning events are surfaced. Done means required adaptive-thinking models can be negotiated and readable reasoning is reliably exposed, with tests covering the event output.

Escrito por el modelo de indexación a partir del texto del issue.

Evaluación

Stack tecnológico
github, vscode
Área
api, devtools
Tipo de issue
Error
Dificultad
4/5
Tiempo estimado
3-5 días
Estado de actividad
Activo
Claridad
Bastante claro
Aptitud para principiantes
42/100

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.