microsoft / microsoft/vscode

Extension-contributed language model providers lose historical thinking needed for vLLM preserve_thinking

Open
#326,597 1 comment 2 reactions 1 assignee Claimed by @vritant24 View on GitHub
Dominant language
TypeScript
Stars
193k
Forks
42.4k
PR merge metrics
PR metrics pending

Description

## Problem

Historical reasoning emitted by a language model provider is not replayed to that provider on the next Copilot turn.

This affects models whose chat templates require prior reasoning to be sent back explicitly, such as Qwen3.6 with `preserve_thinking` and analogous GLM5.2 flows.

## Reproduction

1. Use a language model provider that emits `LanguageModelThinkingPart`.
2. Make a request that produces both reasoning and visible text.
3. Submit a follow-up request in the same Copilot conversation.
4. Inspect the messages received by the contributed provider.

Observed on VS Code 1.129.1 Stable:

- The first response contains reasoning.
- `LanguageModelThinkingPart` is available at runtime.
- The second provider request contains historical assistant text, but no historical `LanguageModelThinkingPart`.
- Therefore the provider cannot forward the prior reasoning as the vLLM assistant `reasoning` field.

## Suspected cause

In the current Copilot extension, `createTurnFromVSCodeChatHistoryTurns(...)` reconstructs an internal `Turn` from `ChatResponseTurn` by flattening response parts to visible text.

The mapping handles markdown, file-tree, content, and anchor parts, but does not handle `ChatResponseThinkingProgressPart`. The resulting `Turn.responseMessage` is text-only, so later prompt construction cannot recover the original reasoning.

This appears to be distinct from issues concerning receiving or displaying thinking tokens. The provider successfully receives and emits thinking; the problem is sending historical thinking back to the provider.

## Expected behavior

When reconstructing public response history, Copilot should preserve `ChatResponseThinkingProgressPart` or otherwise retain enough structured information for the later generic language-model endpoint conversion to produce a `LanguageModelThinkingPart`.

The contributed provider should then receive the same historical thinking that was emitted in the prior response.

VS Code core appears to preserve and serialize thinking correctly, and the generic Copilot endpoint can transport thinking when it is supplied. The loss occurs earlier during public-history-to-internal-turn reconstruction.

Related API-contract reference: #246993.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.