microsoft / microsoft/vscode

fix: surface filtered LM responses as LanguageModelError (fixes #329425)

Open
#329,432 1 comment 0 reactions 0 assignees View on GitHub
agentic-workflows errors-fix
Dominant language
TypeScript
Stars
193k
Forks
42.4k
PR merge metrics
PR metrics pending

Description

### Summary

When an extension calls the VS Code Language Model API through Copilot and the model response is filtered by the Responsible AI Service (`ChatFetchResponseType.Filtered` / `PromptFiltered`), `CopilotLanguageModelWrapper._provideLanguageModelResponse` fell through to a generic `throw new Error(result.reason)` (`"Response got filtered."`). Content filtering is an *expected* outcome, but a bare `Error` thrown from this API path is recorded by the extension host as an `unhandlederror`, producing telemetry noise across Linux/Mac/Windows on Agents product. The fix surfaces filtered responses as a properly-typed `vscode.LanguageModelError.Blocked`, matching how the other expected non-success outcomes (ExtensionBlocked, QuotaExceeded, RateLimited) are already handled.

Fixes microsoft/vscode\#329425
Recommended reviewer: `@lramos15`

### Culprit Commit

| Field | Value |
|-------|-------|
| Commit | Not identified — pre-existing |
| Author | n/a |
| PR | n/a |
| Message | n/a |
| Why | The generic `throw new Error(result.reason)` fall-through for non-success response types has existed since the wrapper's error-handling block was written; no recent regression commit in the shipped range introduced the filtered-response path. This is a pre-existing gap in error typing rather than a regression. The most recent commits touching the file (Auto model adoption, encrypted-thinking, truncated-response handling) do not alter the `result.type !== Success` branch that handles filtered responses. |

### Code Flow

```mermaid
sequenceDiagram
participant Ext as Consuming extension
participant Wrapper as CopilotLanguageModelWrapper
participant Fetch as makeChatRequest2
participant Host as Extension Host

Ext->>Wrapper: provideLanguageModelResponse(...)
Wrapper->>Fetch: makeChatRequest2(...)
Fetch-->>Wrapper: result.type = Filtered (RAI)
Note over Wrapper: ⚠️ Root cause:
Filtered/PromptFiltered fall through
to throw new Error(result.reason)
Wrapper-->>Host: throw new Error('Response got filtered.')
Note over Host: 💥 Recorded as
unhandlederror-Response got filtered.
```

### Affected Files

| File | Role | Evidence |
|------|------|----------|
| `extensions/copilot/src/extension/conversation/vscode-node/languageModelAccess.ts` | root cause / crash site | L844-L864: `if (result.type !== ChatFetchResponseType.Success) { ... } throw new Error(result.reason);` — Filtered/PromptFiltered had no branch and fell through to the generic throw |
| `extensions/copilot/src/platform/chat/common/commonTypes.ts` | supporting | L492 `getFilteredMessage(category, supportsMarkdown)` produces the user-facing filtered message; L447-L453 already classify Filtered/PromptFiltered as `ChatErrorLevel.Info` / `responseIsFiltered`, confirming filtering is an expected, non-error outcome |

### Repro Steps

1. From any extension, call the VS Code Language Model API (`model.sendRequest(...)`) via a Copilot-provided model.
2. Send a prompt/response that triggers Responsible AI content filtering (e.g. output matching public code with copyright filtering enabled).
3. The request rejects with a generic `Error: Response got filtered.` instead of a typed `LanguageModelError`, and the extension host logs it as `unhandlederror`.

### How the Fix Works

**Chosen approach** (`languageModelAccess.ts`): Add an explicit branch for `ChatFetchResponseType.Filtered` and `ChatFetchResponseType.PromptFiltered` in the existing `result.type !== Success` block, throwing `vscode.LanguageModelError.Blocked(getFilteredMessage(result.category, false))`. This fixes the error at the point where the outcome is produced and classified — not by wrapping the crash site in try/catch or swallowing the error. Filtered responses are a known, expected API outcome (the codebase already treats them as `ChatErrorLevel.Info` in `getErrorDetailsFromChatFetchError`), so surfacing them as a typed `LanguageModelError` — consistent with the sibling `ExtensionBlocked`/`QuotaExceeded`/`RateLimited` branches — lets consuming extensions handle filtering via the documented API surface while removing the `unhandlederror` telemetry noise. `getFilteredMessage` is passed `supportsMarkdown = false` because the message flows through an `Error` string, not a rendered chat surface.

**Alternatives considered**: (a) Wrapping the `throw` in a try/catch or downgrading to a log-only path — rejected because it would hide the outcome from consuming extensions, which need to know the response was filtered. (b) Keeping the generic `Error` but renaming it (like `ChatRateLimited`) — rejected because `LanguageModelError` is the documented, typed API contract (`NoPermissions`/`Blocked`/`NotFound`) that extensions catch; a bare renamed `Error` is not part of that contract.

### Recommended Owner

`@lramos15` — top recent contributor to `languageModelAccess.ts` (most-frequent author over the last 90 days, latest commit 2026-08-05), holds Security Maintainer (write+) access to `microsoft/vscode`, and owns the language-model wrapper / model-picker area. The GDPR telemetry owner listed in the file (`jrieken`) was considered first but has no commits to `microsoft/vscode` in the last 90 days (last commit 2026-05-05), failing the liveness gate.

> Generated by [errors-fix](https://github.com/microsoft/vscode-engineering/actions/runs/31114053571) · opus48 · 480.6 AIC · ⌖ 18.2 AIC · ⊞ 18.6K · [◷](https://github.com/search?q=repo%3Amicrosoft%2Fvscode+%22gh-aw-workflow-id%3A+errors-fix%22&type=pullrequests)

---

> [!NOTE]
> This was originally intended as a pull request, but PR creation failed. The changes have been pushed to the branch [`fix/lm-filtered-response-329425-77c272d1713b4fd1`](https://github.com/vscodebot-pr/vscode/tree/fix/lm-filtered-response-329425-77c272d1713b4fd1).
>
> **Original error:** ERR_API: [2026-08-06T15:21:58.724Z] create pull request in microsoft/vscode failed (attempt 1)

Original error: Validation Failed: {"resource":"PullRequest","code":"custom","field":"fork_collab","message":"fork_collab Fork collab can't be granted by someone without permission"} - https://docs.github.com/rest/pulls/pulls#create-a-pull-request
Retryable: false
Suggestion: This error cannot be resolved by retrying. Please check the error details and fix the underlying issue.

To create the pull request manually:

```sh
gh pr create --title "fix: surface filtered LM responses as LanguageModelError (fixes #329425)" --base main --head vscodebot-pr:fix/lm-filtered-response-329425-77c272d1713b4fd1 --repo microsoft/vscode
```

Show patch preview (41 of 41 lines)

```diff
From 96dff1fea5d0c8b67ea8a72c6c04a72e508d236d Mon Sep 17 00:00:00 2001
From: "github-actions[bot]"
Date: Thu, 6 Aug 2026 15:13:19 +0000
Subject: [PATCH] fix: surface filtered LM responses as LanguageModelError
(fixes #329425)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
---
.../conversation/vscode-node/languageModelAccess.ts | 8 +++++++-
1 file changed, 7 insertions(+), 1 deletion(-)

diff --git a/extensions/copilot/src/extension/conversation/vscode-node/languageModelAccess.ts b/extensions/copilot/src/extension/conversation/vscode-node/languageModelAccess.ts
index eedab16ad60..deaa2e43fb1 100644
--- a/extensions/copilot/src/extension/conversation/vscode-node/languageModelAccess.ts
+++ b/extensions/copilot/src/extension/conversation/vscode-node/languageModelAccess.ts
@@ -9,7 +9,7 @@ import * as vscode from 'vscode';
import { IAuthenticationService } from '../../../platform/authentication/common/authentication';
import { CopilotToken } from '../../../platform/authentication/common/copilotToken';
import { IBlockedExtensionService } from '../../../platform/chat/common/blockedExtensionService';
-import { ChatFetchResponseType, ChatLocation, getErrorDetailsFromChatFetchError } from '../../../platform/chat/common/commonTypes';
+import { ChatFetchResponseType, ChatLocation, getErrorDetailsFromChatFetchError, getFilteredMessage } from '../../../platform/chat/common/commonTypes';
import { ConfigKey, IConfigurationService } from '../../../platform/configuration/common/configurationService';
import { getTextPart } from '../../../platform/chat/common/globalStringUtils';
import { EmbeddingType, getWellKnownEmbeddingTypeInfo, IEmbeddingsComputer } from '../../../platform/embeddings/common/embeddingsComputer';
@@ -858,6 +858,12 @@ export class CopilotLanguageModelWrapper extends Disposable {
const err = new Error(result.reason);
err.name = 'ChatRateLimited';
throw err;
+ } else i
... (truncated)
```

Contributor guide

Open the contributing guide

Research direction

Start in extensions/copilot/src/extension/conversation/vscode-node/languageModelAccess.ts, especially the non-success handling in _provideLanguageModelResponse. Read getFilteredMessage in extensions/copilot/src/platform/chat/common/commonTypes.ts and compare the existing typed error branches. Done means filtered responses use the documented typed API outcome instead of the generic error and no longer produce unhandlederror telemetry.

Written by the indexing model from the issue text.

Assessment

Tech stack
typescript
Domain
api
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.