microsoft / microsoft/vscode

Copilot Chat otel.captureContent:false not honored on spans — chat spans carry full prompts/system instructions/responses (copilot-chat 0.57.0 in core)

Open
#326,254 2 comments 1 reaction 1 assignee Claimed by @zhichli View on GitHub
Dominant language
TypeScript
Stars
193k
Forks
42.4k
PR merge metrics
PR metrics pending

Description

Type: Bug

## `github.copilot.chat.otel.captureContent: false` is not honored on the span path — chat spans carry full prompts, system instructions, and responses

### Summary

With OTel emission enabled and `captureContent` explicitly `false` (the default), **span** attributes still include complete conversation content. The log and metric paths honor the setting; the span path does not. The setting's description says: "Capture input/output messages, system instructions, and tool definitions in OTel telemetry. **Contains potentially sensitive data.**" — so users reasonably rely on `false` to keep code and prompts out of their telemetry backend.

### Environment

- Copilot Chat **0.57.0** (built into VS Code core)
- VS Code stable, Windows 11 (10.0.26200)
- Settings:
```json
"github.copilot.chat.otel.enabled": true,
"github.copilot.chat.otel.exporterType": "otlp-grpc",
"github.copilot.chat.otel.protocol": "grpc",
"github.copilot.chat.otel.otlpEndpoint": "http://127.0.0.1:4317",
"github.copilot.chat.otel.captureContent": false
```
- No `COPILOT_OTEL_CAPTURE_CONTENT` environment variable set (no enterprise policy either).

### Repro

1. Configure as above, pointing at any OTLP collector (we used otelcol-contrib 0.156.0 with a file exporter).
2. Reload the window; run any chat request.
3. Inspect the exported `chat` spans.

### Observed

Every `chat` span carries, with `captureContent: false`:

- `copilot_chat.user_request` — the complete user prompt (6,505 chars in our repro)
- `gen_ai.system_instructions` — full system instructions (1,162 chars)
- `gen_ai.input.messages` — the complete message array
- `gen_ai.output.messages` — the full assistant response
- `gen_ai.tool.call.arguments`, `gen_ai.tool.call.result`, `gen_ai.tool.definitions`

Log records and metrics in the same capture correctly contain metadata only (token counts, model, timing) — the gating appears to be applied on those paths but missing on the span-building path.

### Expected

With `captureContent: false`, span attributes contain no message content, no system instructions, no tool arguments/results — matching the log/metric behavior and the setting's description.

### Impact

Anyone who enabled OTel spans while relying on `captureContent: false` is shipping full conversation content (which can include proprietary source code) to their telemetry backend without knowing it. For our part we've had to exclude span capture from all customer-adjacent environments until this is fixed — which is a pity, because the span path is the only place cache token detail (`gen_ai.usage.cache_read.input_tokens`) and `copilot_chat.copilot_usage_nano_aiu` are emitted, and those are exactly the attributes cost-observability setups want.

Happy to provide sanitized span samples on request.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.