Copilot managed OTel policy can be missed during extension activation
- Dominant language
- TypeScript
- Stars
- 193k
- Forks
- 42.4k
- PR merge metrics
- PR metrics pending
Description
## Summary
Enterprise-managed Copilot OpenTelemetry settings can be missed during VS Code startup. The configuration model eventually reports the managed values correctly, but the already-running Copilot extension can remain on its startup fallback for the lifetime of that extension host and export no enterprise telemetry.
The outcome is nondeterministic and is easier to reproduce when many windows are restored concurrently or when managed-settings resolution is delayed.
## Managed configuration
For example:
```json
{
"telemetry": {
"enabled": true,
"endpoint": "https://collector.example.com",
"protocol": "http/json",
"captureContent": true,
"lockCaptureContent": true,
"serviceName": "github-copilot"
}
}
```
## Observed behavior
A successful activation logs both the extension exporter and the embedded Copilot runtime exporter:
```text
[OTel] Instrumentation enabled — exporter=otlp-http endpoint=https://collector.example.com/ captureContent=true
[CopilotCLI] OpenTelemetry enabled: exporter=otlp-http, source=github.copilot
```
A failed activation logs only:
```text
[CopilotCLI] OpenTelemetry enabled: exporter=file, source=github.copilot
```
There is no warning or OTLP connection attempt. Later, **Developer: Policy Diagnostics** reports the managed OTel values as successfully applied even though that extension host is still not exporting to the managed endpoint.
Restarting only the extension host after policy has resolved generally makes the next activation use the managed exporter.
## Root cause
Two startup paths are not ordered.
### Late policy-reference registration
`PolicyConfiguration.initialize()` resolves keys registered at initial configuration startup. Extension-contributed settings are registered later. Their policy references are processed by an async listener in:
`src/vs/platform/configuration/common/configurations.ts`
```ts
this._register(this.defaultConfiguration.onDidChangeConfiguration(
async ({ properties }) => this.update(await this.updatePolicyDefinitions(properties), true)));
```
The event source does not await the listener's promise before extension activation continues.
### One-shot OTel service construction
Copilot resolves its OTel configuration once during service registration in:
`extensions/copilot/src/extension/extension/vscode-node/services.ts`
```ts
const otelSettings = workspace.getConfiguration('github.copilot.chat.otel');
const policyValue = key => otelSettings.inspect(key)?.policyValue;
const otelConfig = resolveOTelConfig(/* ... */);
```
It then permanently registers either `NodeOTelService` or `InMemoryOTelService` for that extension-host activation.
If the late policy references have not resolved yet, all `policyValue` reads are `undefined`. The service is constructed without enterprise OTel enabled. When policy arrives later, the VS Code configuration model updates, but the OTel service is not recreated or reconfigured.
The reload watcher in `extensions/copilot/src/extension/otel/vscode-node/otelContrib.ts` does not reliably recover this case. It is registered after service construction and snapshots the then-current values. Policy can resolve after the service read but before the watcher snapshot, leaving the service stale without a later change to trigger a reload prompt.
## Why `exporter=file` appears
This is the expected fallback in:
`extensions/copilot/src/extension/chatSessions/copilotcli/node/copilotcliSessionService.ts`
When the frozen extension OTel config says telemetry was not explicitly enabled, VS Code keeps the embedded runtime's debug tracker alive by setting `COPILOT_OTEL_FILE_EXPORTER_PATH` to the OS null device and deleting inherited OTLP endpoints. The runtime then correctly reports `exporter=file`; it is not selecting the enterprise collector.
## Expected behavior
When managed telemetry is enabled, every extension host must resolve and apply that policy before accepting telemetry-producing Copilot work.
At minimum:
- Copilot must not permanently construct a disabled OTel service while managed OTel policy is still resolving.
- The extension and embedded Copilot runtime must receive the same effective managed telemetry configuration.
- Policy Diagnostics must distinguish configuration that is merely resolved from the exporter actually active in the extension host/runtime.
- A managed collector must not silently degrade to the null-file fallback.
## Suggested implementation direction
The core-owned `chat.agentHost.otel.*` policy settings are registered before initial policy resolution. The late `github.copilot.chat.otel.*` settings are references to those same policies.
A targeted fix can have the extension fall back to the corresponding core policy-owner `policyValue` when its own late policy-reference value is missing. This should cover `enabled`, exporter type, wire protocol, endpoint, capture content, service name, resource attributes, and headers without importing user values from the core settings.
The embedded runtime path should also defer its telemetry decision until managed telemetry has resolved, rather than deriving a permanent `/dev/null` environment from a potentially stale extension snapshot.
A broader core fix would expose an awaitable barrier for late policy-definition registration and ensure relevant extensions do not activate before it settles.
## Deterministic reproduction
1. Configure server-managed `telemetry.enabled`, `telemetry.endpoint`, and `telemetry.captureContent`.
2. Do not set local OTel environment variables or file-based managed settings.
3. Delay the managed-settings/policy-definition response until after Copilot service registration.
4. Activate Copilot Chat.
5. Allow policy resolution to complete.
6. Run **Developer: Policy Diagnostics**.
**Actual:** diagnostics show the policy values, but the Copilot output has no `[OTel] Instrumentation enabled` line and the embedded runtime reports `exporter=file`.
**Expected:** the extension and embedded runtime both initialize against the managed OTLP endpoint, or Copilot waits until policy resolution has completed.
## Acceptance criteria
- Managed OTel policy is applied deterministically across cold starts, window restores, and extension-host restarts.
- Copilot does not emit telemetry-producing work before mandatory OTel policy is resolved.
- Direct extension spans and embedded runtime spans use the same managed endpoint, protocol, capture posture, service name, resource attributes, and headers.
- A policy arriving after initial service registration either initializes the exporter safely or triggers a deterministic extension-host restart before Copilot work proceeds.
- Failed or unresolved mandatory policy produces an actionable warning instead of silently selecting the null-file fallback.
- Policy Diagnostics reports the live extension/runtime exporter and any unresolved or stale-policy state.
- Automated coverage deterministically delays late policy-reference registration past extension service construction and verifies recovery.
Contributor guide
Assessment
This issue has not been assessed yet.