github / github/copilot-cli

OTel spans for subagent calls omit billing attributes (github.copilot.nano_aiu, github.copilot.cost), so external cost accounting undercounts actual billing

Ouverte
#4,224 4 commentaires 1 réaction 0 personnes assignées Voir sur GitHub
area:agents
Langage dominant
Shell
Étoiles
11.2k
Forks
1.9k
Merge moyen
14 h 16 min
PR mergées (30 j)
6

Description

### Describe the bug

When a session delegates work to subagents (the `task` tool / custom agents), the OTel spans for those subagent model calls omit **all** billing attributes. The subagent calls consume real AI credits (per the cumulative session display — see #4207, which notes session usage "including usage by the main agent, subagents, and background operations"), but that consumption is invisible to OpenTelemetry, so any external cost accounting built on the documented OTel telemetry (`copilot help monitoring`) systematically undercounts actual billing.

Main-agent `chat` spans carry the full billing envelope:

- `github.copilot.nano_aiu`
- `github.copilot.cost`
- `github.copilot.initiator` (`user` / `agent`)
- `github.copilot.interaction_id` / `github.copilot.turn_id`

Subagent model calls emit spans with correct `gen_ai.*` semconv attributes (`gen_ai.operation.name`, `gen_ai.request.model`, `gen_ai.usage.input_tokens` / `output_tokens` / `cache_read.input_tokens` / `cache_creation.input_tokens`) but carry **none** of the `github.copilot.*` billing attributes — no `nano_aiu`, no `cost`, not even `initiator` or an interaction id. This holds at every level:

1. The subagent's child `chat` spans: no billing attributes.
2. The subagent's own nested `invoke_agent` span (which does carry `gen_ai.agent.name` and rolled-up token counts): no billing attributes.
3. The session root `invoke_agent` span's `github.copilot.nano_aiu` rollup equals the sum of the main-agent `chat` spans' `nano_aiu` **exactly** — i.e. the subagent spend is not folded in anywhere.

Observed within a single trace from one session (models are our configured custom-agent models; the same holds for built-in `task` delegation):

| Span | Model | `gen_ai.usage.input_tokens` | `nano_aiu` | `cost` |
|---|---|---|---|---|
| root `invoke_agent` | claude-fable-5 | 1,963,612 | 506,956,900,000 | 13 |
| main-agent `chat` × 13 | claude-fable-5 | … | present on every one; sums to exactly 506,956,900,000 | 1 each |
| nested `invoke_agent` (custom agent A) | claude-haiku-4.5 | 296,978 | **absent** | **absent** |
| nested `invoke_agent` (custom agent B) | gpt-5.6-terra | 155,061 | **absent** | **absent** |
| subagent `chat` spans | claude-haiku-4.5 / gpt-5.6-terra | 124K–321K each | **absent** | **absent** |

In our sessions the unreported subagent calls account for roughly 10–15% of estimated session cost (rate-card priced from the token counts, which *are* reported).

### Affected version

GitHub Copilot CLI 1.0.73

### Steps to reproduce the behavior

1. Enable OTel export (`COPILOT_OTEL_ENABLED` with an OTLP endpoint, or `COPILOT_OTEL_FILE_EXPORTER_PATH`).
2. Run an interactive session and issue a prompt that delegates to a subagent — e.g. a custom agent configured with a different model, or anything that triggers the `task` tool.
3. Inspect the exported spans for the session trace.
4. Main-agent `chat` spans carry `github.copilot.nano_aiu` / `github.copilot.cost`; the subagent `invoke_agent` and `chat` spans carry token usage but no billing attributes, and the root rollup excludes them.

### Expected behavior

Subagent model calls should carry the same billing attributes as main-agent calls (`github.copilot.nano_aiu`, `github.copilot.cost`), so that OTel-based cost accounting matches what is actually billed. Alternatively, if this is intentional (subagent calls genuinely bill zero credits), it would be great to have that documented in `copilot help billing` / the monitoring docs — though the cumulative session credit display suggests they are billed.

### Additional context

- OS: Windows 11, x86_64, Windows Terminal / PowerShell (also reproduced from sessions on the same version exporting via OTLP to an OpenTelemetry Collector).
- Related issues: #4207 (per-subagent credit breakdown in `/usage` — confirms subagent usage is part of session credit consumption), #1582 (`/usage` undercounts quota consumption; suspects background operations), #2068 (background compaction consumes a premium request), #4107 (verified `nano_aiu` matches the interactive footer exactly for main-agent sessions — the mismatch reported here only appears once subagents are involved), #3778 (OTel cost metric feature request).
- Happy to provide sanitized span exports (main-agent vs subagent, same trace) on request.

Guide de contribution

Ouvrir le guide de contribution

Piste de recherche

Commencez par la télémétrie de Copilot CLI documentée par `copilot help monitoring`, puis reproduisez le problème à l’aide de l’outil `task` ou d’un agent personnalisé avec l’export OTel activé. Comparez les spans `chat` de l’agent principal avec les spans `invoke_agent` et `chat` du sous-agent, y compris le session root rollup. Le travail est considéré comme terminé lorsque les attributs de facturation et l’utilisation du sous-agent sont représentés de manière cohérente, ou que le comportement de facturation est documenté si ces appels sont intentionnellement gratuits.

Rédigé par le modèle d'indexation à partir du texte de l'issue.

Évaluation

Stack technique
github
Domaine
cli, observability
Type d'issue
Bug
Difficulté
4/5
Temps estimé
3-5 jours
Activité
Active
Clarté
Plutôt claire
Accessibilité débutants
48/100

Recevez les nouvelles issues par e-mail

Un résumé court des issues GitHub adaptées aux débutants.