microsoft / microsoft/vscode

Add telemetry for main MCP failure modes

Open
#336,796 0 comments 0 reactions 1 assignee Assigned to @sandy081 View on GitHub
Dominant language
TypeScript
Stars
193k
Forks
42.4k
PR merge metrics
PR metrics pending

Description

## Problem

We do not have enough VS Code-side telemetry to quantify the main MCP failure modes and drive quality improvements with the SDK/runtime teams.

Today, `mcp/serverBootState` records only the final state and elapsed time. It does not identify the failure stage or a privacy-safe failure category. Successful initialization is covered by `mcp/serverBoot`, but failures during startup, capability discovery, and tool execution cannot be reliably separated or trended.

## Proposal

Add VS Code telemetry for the MCP lifecycle's major failure points:

- Server startup and connection: launch/resolve, transport connection, protocol initialization, authentication/user interaction, and capability discovery.
- Tool execution: invocation rejected before dispatch, transport/request failure, timeout, cancellation, invalid request/response or schema, server-returned error, and tool catalog changes.

Record only categorical, privacy-safe dimensions, such as:

- Lifecycle stage and normalized failure category/code.
- Transport type and configuration source/scope.
- Whether the operation was retryable, retried, or recovered.
- Time to failure and, for tool calls, execution duration.
- SDK/runtime ownership boundary when known, so regressions can be routed to the responsible team.

Do not collect raw error messages, command lines, URLs, tool arguments/results, server-provided content, or other user data.

## Outcome

Dashboards should make it possible to answer:

- What percentage of MCP server starts and tool calls fail?
- Which failure categories and lifecycle stages account for most failures?
- How do rates differ by transport and configuration source?
- Which failures originate in VS Code versus the SDK/runtime?
- Are fixes improving failure and recovery rates over time?

This can extend the existing `mcp/serverBootState` event and add a corresponding tool-call outcome event, or use new events if that produces cleaner schemas.

Contributor guide

Open the contributing guide

Research direction

Start by locating the existing mcp/serverBootState and mcp/serverBoot telemetry definitions and tracing the MCP lifecycle through startup, discovery, and tool execution. Define privacy-safe schemas that distinguish the listed stages and failure categories without collecting user data. Done means dashboards can answer the stated failure, transport, ownership, retry, and recovery questions.

Written by the indexing model from the issue text.

Assessment

Tech stack
typescript
Domain
observability
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.