microsoft / microsoft/vscode

BYOK custom endpoint (Responses API) sends truncated input to servers that do not store responses

Open
#325,436 1 comment 3 reactions 1 assignee Claimed by @vritant24 View on GitHub
Dominant language
TypeScript
Stars
193k
Forks
42.4k
PR merge metrics
PR metrics pending

Description

BYOK custom endpoints with `apiType: "responses"` against a server that does not persist responses (echoes `store: false`) still record the response id as a stateful marker. The next request's input is then truncated to the items after the marker, and a later guard strips the non-`resp_` `previous_response_id`, so the server receives an unanswerable request: a `tool_result` without its `tool_use`, or no user message at all, rejected with a 400.

**Cause:** `OpenAIResponsesProcessor.processChunk` (extensions/copilot/src/platform/endpoint/node/responsesApi.ts) emits the stateful marker from `response.completed` unconditionally; the response's `store` field is read nowhere. Separately, the BYOK Responses branch of `OpenAIEndpoint.createRequestBody` strips a non-`resp_` `previous_response_id` only after the input was already truncated to the marker, so the truncation and chaining decisions can disagree.

**Repro:**
1. Configure a `customendpoint` provider with `"apiType": "responses"` against a stateless Responses-compatible server (echoes `store: false`, response ids not prefixed `resp_`).
2. Run a tool-calling turn, then send any follow-up in the same conversation.
3. The follow-up carries input truncated to post-marker items with no `previous_response_id`; the server (or its upstream) rejects it, e.g. `tool_result` without a matching `tool_use`.

Reproduces on `main` (code path present at HEAD).

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.