microsoft / microsoft/vscode

# [copilot] convertToApiChatMessage silently drops Document content parts — PDF attachments never reach LanguageModelChatProvider extensions

Open Beginner friendly
#336,694 2 comments 0 reactions 1 assignee View on GitHub

Nobody has claimed this yet.

Dominant language
TypeScript
Stars
193k
Forks
42.7k
PR merge metrics
PR metrics pending

Description

[copilot] convertToApiChatMessage silently drops Document content parts — PDF attachments never reach LanguageModelChatProvider extensions

Type: Bug
Area: GitHub Copilot Chat extension (extensions/copilot), Language Model API
Related: #324961 (BYOK PDF support requires a hardcoded model family)
VS Code version: 1.139
Copilot Chat extension: bundled with 1.139

Summary

When a chat request includes a PDF attachment, the Copilot prompt pipeline renders it as a Raw.ChatCompletionContentPartKind.Document content part — but convertToApiChatMessage in the extension bridge has no branch for Document, so the part is silently dropped during conversion to vscode.LanguageModelChatMessage. Third-party LanguageModelChatProvider extensions therefore never receive the PDF bytes (neither as a LanguageModelDataPart nor anywhere else). The <attachments> text wrapper still appears in the message text, but it is empty.

Steps to Reproduce

  1. Register a minimal chat provider:
vscode.lm.registerLanguageModelChatProvider('repro', {
    provideLanguageModelChatInformation() {
        return [{
            id: 'pdf-test',
            name: 'pdf-test',
            // gpt-5* family is required to pass the family gate in
            // modelSupportsPDFDocuments (see related issue #324961)
            family: 'gpt-5.4',
            version: '1',
            capabilities: { toolCalling: true, imageInput: true },
            maxInputTokens: 128000,
            maxOutputTokens: 16000,
        }];
    },
    async provideLanguageModelChatResponse(model, messages, options, progress, token) {
        for (const m of messages) {
            for (const p of m.content) {
                console.log(Object.keys(p));
            }
        }
        progress.report(new vscode.LanguageModelTextPart('ok'));
    },
});
  1. Select the model in the chat picker (Agent mode), drag a PDF file into the chat input, send any prompt.
  2. Inspect messages in the provider.

Expected: the PDF arrives as a LanguageModelDataPart with mimeType: 'application/pdf' (analogous to how images arrive as data parts), because fileVariable.tsx explicitly renders PDFs as <Document> prompt elements.

Actual: no data part at all. The user message's text contains an empty <attachments></attachments> wrapper — the Document part was rendered by the prompt pipeline and then silently discarded.

Root Cause

extensions/copilot/src/platform/endpoint/vscode-node/extChatEndpoint.ts, convertToApiChatMessage (around lines 356-391):

for (const contentPart of message.content) {
    if (contentPart.type === Raw.ChatCompletionContentPartKind.Text) { ... }
    else if (contentPart.type === Raw.ChatCompletionContentPartKind.Image) { ... }
    else if (contentPart.type === Raw.ChatCompletionContentPartKind.CacheBreakpoint) { ... }
    else if (contentPart.type === Raw.ChatCompletionContentPartKind.Opaque) { ... }
    // <- no branch for Raw.ChatCompletionContentPartKind.Document
}

PDFs enter this function because extensions/copilot/src/extension/prompts/node/panel/fileVariable.tsx renders them as:

<Document data={base64string} mediaType='application/pdf' />

(only when supportsVision && modelSupportsPDFDocuments(endpoint) — the family gate covered by #324961).

Document is clearly a first-class content part elsewhere: renderedMessageToTsxChildren in extensions/copilot/src/extension/prompts/node/agent/agentPrompt.tsx (~lines 581-582) explicitly handles Raw.ChatCompletionContentPartKind.Document for history replay. The conversion function simply misses it.

Suggested Fix

Mirror the Image branch in convertToApiChatMessage:

} else if (contentPart.type === Raw.ChatCompletionContentPartKind.Document) {
    apiContent.push(
        new vscode.LanguageModelDataPart(
            Buffer.from(contentPart.documentData.data, 'base64'),
            contentPart.documentData.mediaType
        )
    );
}

This would let extension providers receive PDF attachments the same way the built-in GPT/Claude endpoints do, gated by their own declared capabilities.

Impact

Any third-party LanguageModelChatProvider (BYOK / custom vendor extensions) cannot receive PDF attachments at all in 1.139, regardless of model capability — a hard blocker for document Q&A scenarios that work fine for built-in models.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in extensions/copilot/src/platform/endpoint/vscode-node/extChatEndpoint.ts at convertToApiChatMessage, then compare the existing Image branch with Document handling in extensions/copilot/src/extension/prompts/node/agent/agentPrompt.tsx and fileVariable.tsx. Reproduce with the minimal provider and a PDF attachment; done means the provider receives the PDF as a LanguageModelDataPart with its application/pdf media type.

Written by the indexing model from the issue text.

Assessment

Tech stack
typescript, vscode
Domain
api
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
84/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.