BYOK Chat Completions: historical reasoning_content dropped across turns (Custom Endpoint & extension providers; Kimi K3 / GLM / DeepSeek)
- Dominant language
- TypeScript
- Stars
- 193k
- Forks
- 42.4k
- PR merge metrics
- PR metrics pending
Description
Copilot Chat Extension Version: (1.134.0)
VS Code Version: ((1.134.0)
OS Version: (Windows 11)
Feature: agent mode (panel chat)
Selected model: BYOK Custom Endpoint — Kimi K3 (apiType: 'chat-completions', thinking: true); also reproducible with GLM-4.7/5.x and DeepSeek thinking mode, and with extension-contributed LanguageModelChatProviders
Logs: (attach Chat debug log / request log showing the second-turn request body missing reasoning_content on the previous assistant message)
Steps to Reproduce:
Configure a Custom Endpoint model pointing at a Kimi K3 (or GLM/DeepSeek thinking) Chat Completions endpoint, with thinking: true and tool calling enabled.
Ask a question that triggers at least one tool call and produces reasoning (first turn completes normally — reasoning is echoed within the turn).
Ask a follow-up question in the same chat session.
Observe the outgoing request for the follow-up: the previous assistant message no longer carries reasoning_content. Kimi K3 / kimi-k2.7-code require it; GLM (clear_thinking: false) requires it; DeepSeek thinking mode returns HTTP 400 when tools are present without it.
Problem
When using a BYOK model over Chat Completions with thinking enabled — whether via the built-in Custom Endpoint provider or via an extension-contributed LanguageModelChatProvider — reasoning from previous conversation turns is silently dropped from the outgoing request:
First turn works — within a single agent turn, multiple thinking/tool-call iterations do carry reasoning content (the current-turn path is not gated).
Second turn loses it — as soon as a turn becomes history, the previous assistant message's thinking is stripped before rendering, so the next request contains no reasoning_content for prior turns.
This affects both entry points:
Custom Endpoint (built-in BYOK): the history gate in the prompt renderer excludes chat-completions entirely (details below).
Extension-contributed providers: thinking deltas from these models typically arrive without an id, and id-less thinking parts are skipped when rebuilding the prompt — so cross-turn reasoning never makes it into the request even before the gate is reached.
This breaks providers that require preserved thinking across turns (Kimi K3 / kimi-k2.7-code, GLM with clear_thinking: false), and silently degrades providers that recommend it (DeepSeek thinking mode, which 400s when tools are present without prior reasoning_content).
Root cause
1. History gate excludes Chat Completions — extensions/copilot/src/extension/prompts/node/panel/toolCalling.tsx:
For chat-completions, apiSupportsHistoricalThinking is always false, so includeThinking is false for every historical round → ThinkingDataContainer is never emitted → OpenAIEndpoint.getCompletionsCallback() has nothing to write into reasoning_content.
2. Extension-contributed providers lose id-less thinking parts — extChatEndpoint.ts forwards streamed thinking as id: chunk.id || '', and languageModelAccessPrompt.tsx groupThinkingParts() skips any LanguageModelThinkingPart with no id. Models like Kimi/DeepSeek stream reasoning_content without any id, so their thinking parts never survive into a rebuilt prompt. A generateUuid() fallback (as ThinkingDataItem.createOrUpdate already does in the tool-calling loop) would fix the id-less case.
Provider documentation requiring preserved reasoning_content
Kimi K3 / kimi-k2.7-code — preserved thinking is always-on; multi-turn and tool-call conversations must echo the full assistant message including reasoning_content: https://platform.kimi.com/docs/guide/use-thinking-models#preserved-thinking
GLM (Z.AI) 4.7 / 5.x — with clear_thinking: false, clients "must return the complete, unmodified reasoning_content back to the API"; blocks must match the original sequence: https://docs.z.ai/guides/capabilities/thinking-mode
DeepSeek thinking mode — when the request carries tools, the intermediate assistant's reasoning_content "must participate in the context concatenation and must be passed back to the API in all subsequent user interaction turns" or the API returns 400: https://api-docs.deepseek.com/guides/thinking_mode
Suggested fix
Extend the historical-thinking gate in toolCalling.tsx so chat-completions models that declare thinking support (at minimum the Kimi family; arguably any capabilities.supports.thinking chat-completions endpoint) also pass apiSupportsHistoricalThinking. The write-back side (reasoning_content emission in OpenAIEndpoint.getCompletionsCallback) is already in place from #312746 / #314019.
Add an id fallback in extChatEndpoint.ts (chunk.id || generateUuid()) and/or make groupThinkingParts() tolerate id-less parts, so extension-contributed providers whose models stream reasoning without ids don't lose thinking across turns.
Related
#312746 (fixed: emit reasoning_content on assistant tool-call message)
#314019 (merged: aggregate streamed reasoning parts + emit reasoning_content; explicitly scoped to within-turn rounds, not cross-turn history)
#318933 (synthetic ID for reasoning_content)
Contributor guide
Assessment
This issue has not been assessed yet.