Define a 1P telemetry model for azd extensions
- Dominant language
- Go
- Stars
- 569
- Forks
- 364
- Avg merge
- 2d 19h
- Merged PRs (30d)
- 136
Description
Partner teams (Foundry especially) need richer telemetry from azd extensions. Things like agent ID attribution, scenario tagging, usage metrics. We don't have a safe way to do this today. We should start with a first-party-only model using span enrichment, gated by a capability check on the extension telemetry interface.
### Background
Extension authors want to understand how their extensions are being used in production and some of the data they need isn't available today. Agent IDs, scenario metadata, usage depth, etc., would all enhance the understanding of how azd is being used by customers.
The problem is that extensions don't necessarily run in a trust boundary where we can guarantee compliance around arbitrary telemetry. Opening that up without guardrails creates privacy and compliance exposure that we need to think through carefully.
### Options we talked through
1. **Open telemetry API for all extensions.** Most flexible. Also requires a full governance model (validation, sanitizing, compliance review) we don't have. It's a LOT of compliance work.
2. **First-party telemetry only.** Restrict the capability to Microsoft-owned extensions. Narrower, shippable, and we can learn what the right patterns are before we expand.
3. **Lightweight span enrichment.** Instead of completely custom telemetry, let extensions add attributes to existing spans. Incremental, low-risk.
I think the right path is (2) + (3). First-party gating with span enrichment as the initial capability. Gets partner teams what they need without the design and compliance complexity of a fully open model.
This isn't about limiting extensions long-term. It's about getting the trust model right before we expand scope. Open extension telemetry is harder than it looks:
- Extensions could unintentionally emit sensitive data (raw errors, user content, secrets in diagnostic payloads). We don't have filtering or validation to catch this today.
- There's no mechanism to distinguish trusted vs untrusted telemetry sources. We can't enforce different rules for different extension types.
- Compliance requirements (GDPR, data inventory, retention) apply to everything we collect. Each new data source adds surface area.
Starting with first-party lets us build this in a controlled environment, figure out the patterns, then extend access over time. The goal is to eventually support broader scenarios (including third-party), but with guardrails in place first.
### Example scenarios
To make this concrete, here are a couple hypothetical use cases (to be extended):
1. **Agent ID attribution.** A Foundry extension enriches azd spans with something like `azd.extension.custom.agent_id` so agent invocations can be correlated across the pipeline.
2. **Extension-scoped child spans.** A first-party extension creates child spans under the azd command span to capture duration and outcome of its own work (e.g., a model deployment step), separate from host telemetry.
These are illustrative. We need to follow up with Foundry and other partner teams to validate against their actual scenarios.
## Rough Plan
### 1. Define the enrichment surface
Extension metadata (extension ID, version) should be automatically attached to all extension activity without requiring opt-in from the author. This is baseline context that every consumer of this telemetry will need.
Beyond that, the initial set of custom attributes we'd support on existing spans:
- Agent ID (for Foundry scenarios) and simple attributes
- Scenario scoped spans
We should consider an `azd.extension.custom.*` namespace to keep these OTel-compatible and clearly scoped. The full schema design (versioning, attribute types, cardinality constraints, deprecation process) is an open question for implementation. Worth noting that azd's existing telemetry pipeline requires all attributes to be in the typed fields inventory with GDPR classification and purpose, so any new attributes need to go through that process too.
### 2. Sort out the data model
Two distinct scenarios worth considering, even if we phase them:
- **Span enrichment** is for identity and scenario metadata on azd's own operations. Small, well-defined attributes on existing spans.
- **Child spans** are the right primitive when the extension is doing distinct, timeable work with its own latency and error semantics. Putting everything on the parent span loses causality and makes error attribution ambiguous (whose failure is it?).
Even if child spans are deferred to a later phase, we should document that intent so the v1 enrichment API doesn't get overloaded as a substitute.
### 3. Build the gating mechanism
Add a capability check on the telemetry interface that indicates whether an extension is authorized to use it. Extensions without the capability get a clear signal that custom telemetry is unavailable. Our current approach is to validate whether the extension comes from the blessed 1P registry to decide whether it's trusted.
A few things worth specifying up front:
- **Fail if uncertain.** If we can't validate first-party status (offline, registry unreachable), telemetry enrichment is unavailable. Safer than failing open, even though it means some air-gapped scenarios won't work without additional effort.
- **Diagnostics.** Extension authors need a way to figure out why telemetry access is denied. We should be sure to log why it's failing.
- **Skip signing.** We considered whether we need full extension signing (signed manifests, content hashes at runtime) to prevent spoofing. The threat model is pretty narrow though. An attacker would need to spoof the registry to gain the ability to... send us additional telemetry attributes. Not a high-value target. First-party extensions already ship through Microsoft-controlled pipelines, so supply chain trust is largely inherited. It's worth revisiting if we expand beyond first-party or add higher-privilege capabilities, but not a blocker for this scope.
### 4. Define the first-party onboarding path
We need to document how the 1P story works, expectations for 1P extension authors on the type of telemetry we're willing to collect, the GDPR approval process, surfacing in our dashboards, expected timelines, etc.
### 5. Validate with partners
Review the model with Foundry and any other teams with near-term telemetry needs before shipping. Make sure the initial attribute set actually covers their scenarios.
### 6. Open questions
- What are Foundry's actual telemetry scenarios? We need to validate against real requirements, not assumptions.
- What's the right namespace and schema for enrichment attributes? `azd.extension.custom.*`?
- Is the capability-check gating sufficient, or do we need something more?
- Are there other extension capabilities (beyond telemetry) that need similar privileged access treatment for 1P extensions?
Contributor guide
Assessment
This issue has not been assessed yet.