Telemetry: `agentHost.turnCompleted.permissionLevel` is provider-specific and shouldn't live on the generic turn event
@TylerLeonhardt is already working on this.
Since Jun 30, 2026.
Assessment
This issue has not been assessed yet.
Description
Problem
The agentHost.turnCompleted telemetry event (IAgentHostTurnCompletedEvent / IAgentHostTurnCompletedClassification in src/vs/platform/agentHost/node/agentHostTelemetryReporter.ts) includes a permissionLevel field. Unlike the rest of that event (timeToFirstProgress, totalTime, result, model), permissionLevel is not generic across agent-host providers — it is only ever populated for Copilot AH and is empty for Claude and Codex AH.
Root cause
The value is sourced in shared code from the platform autoApprove config key:
// src/vs/platform/agentHost/node/agentSideEffects.ts (_getTurnTelemetryContext)
const permissionValue = state?.config?.values[SessionConfigKey.AutoApprove];
Only the Copilot provider writes SessionConfigKey.AutoApprove. The other harnesses model permissions/approvals with their own, namespaced config keys:
- Claude —
ClaudeSessionConfigKey.PermissionMode(permissionMode):default/acceptEdits/bypassPermissions/plan/auto - Codex —
CodexSessionConfigKey.ApprovalPolicy(codex.approvalPolicy), pluscodex.sandboxMode,codex.networkAccessEnabled,codex.webSearchMode, etc.
So the shared turn-completed path bakes in a Copilot-specific assumption (autoApprove), and permissionLevel comes back empty for every Claude/Codex turn.
Why it doesn't belong on the generic event
agentHost.turnCompleted is otherwise a harness-agnostic turn performance event. permissionLevel is a provider-specific config snapshot, and there is no lossless normalization across harnesses — Codex's sandboxMode × networkAccess axes and Claude's single permissionMode axis don't map cleanly onto Copilot's default / autoApprove / autopilot. Putting one provider's config field on the shared classification makes the metric look generic when it isn't, and silently yields empty data for 2 of 3 providers.
Proposed direction
- Remove
permissionLevelfrom the sharedagentHost.turnCompletedevent so it stays purely harness-agnostic (timing / result / model). - Emit permission/config telemetry from provider-specific code instead (e.g.
agentHost.copilot.*), so each harness reports its own config faithfully and the values are accurate per provider. This mirrors the provider-namespaced pattern (e.g. the proposedagentHost.claude.subagentResolved).
Internal context / scorecard gap: microsoft/vscode-internalbacklog#8205.
Code references
src/vs/platform/agentHost/node/agentHostTelemetryReporter.ts—permissionLevelonIAgentHostTurnCompletedEvent+ classificationsrc/vs/platform/agentHost/node/agentSideEffects.ts—_getTurnTelemetryContextreadsSessionConfigKey.AutoApprovesrc/vs/platform/agentHost/node/agentHostTurnTracker.ts— plumbspermissionLevelthroughturnStarted→turnCompletedsrc/vs/platform/agentHost/common/sessionConfigKeys.ts,common/claudeSessionConfigKeys.ts,node/codex/codexSessionConfigKeys.ts— the divergent per-provider key namespaces
- Dominant language
- TypeScript
- Stars
- 193k
- Forks
- 42.9k
- PR merge metrics
- PR metrics pending
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from microsoft/vscode
-
testplan-item
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
-
new release
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
-
testplan-item
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 84/100
-
testplan-item
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
All issues in microsoft/vscode
Similar issues
-
clawsweeper:fix-shape-clear clawsweeper:queueable-fix clawsweeper:source-repro impact:ux-friction issue-rating: 🦞 diamond lobster no-stale P3
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
-
community first-timers-only good first issue hacktoberfest help wanted low hanging fruit up-for-grabs
Difficulty 1/5 Under an hour Newbie friendliness 76/100
-
code-quality refactoring
Difficulty 2/5 1-3 hours Newbie friendliness 84/100
github/gh-aw-firewall#8816 ·
-
integration:quickjs org:external priority:backlog topic:code-interpreter topic:middleware type:feature
Difficulty 2/5 1-3 hours Newbie friendliness 74/100
langchain-ai/deepagents#6450 ·
-
Difficulty 1/5 Under an hour Newbie friendliness 88/100
vercel/react-tweet#225 ·