github / github/copilot-cli

--output-format json omits token/cost usage that OTel exposes

Đang mở
#4,107 0 bình luận 0 reaction 0 người được giao Xem trên GitHub
area:models area:non-interactive
Ngôn ngữ chính
Shell
Star
11.2k
Fork
1.9k
Merge trung bình
14 giờ 16 phút
Pull request đã merge (30 ngày)
6

Mô tả

### Describe the feature or problem you'd like to solve

_No response_

### Proposed solution

`--output-format json`'s terminal `result` event only carries legacy fields (`premiumRequests`, `totalApiDurationMs`, `sessionDurationMs`, `codeChanges`) under `usage`. It does not include token counts (input/output/cached) or AI-credit cost, even though this exact data is computed internally during the very same run and is fully available via OpenTelemetry (`COPILOT_OTEL_ENABLED` / `COPILOT_OTEL_FILE_EXPORTER_PATH`, documented in `copilot help monitoring`).

I verified with raw, unmodified CLI output (happy to attach the files):
- A full, unfiltered JSONL dump of `-p ... --output-format json` shows the only `usage`-bearing event is the terminal `result`, and its `usage` object has no token/cost fields at all.
- Enabling the OTel file exporter *simultaneously* with `--output-format json` (same invocation) proves the CLI does compute `gen_ai.usage.input_tokens` / `output_tokens` / `cache_read.input_tokens` / `cache_creation.input_tokens` and `github.copilot.nano_aiu` (exact AI-credit cost) during the run - it's just never written into the `result` JSON event.
- Cross-checked `nano_aiu`-derived AI Credits against the interactive footer ("AI Credits X.X") in the same session - they match exactly, confirming this is real billing data, not an estimate.

Proposed solution: add `inputTokens`, `outputTokens`, `cacheReadInputTokens`, `cacheCreationInputTokens`, and `aiCredits`/`costUSD` (broken down per model if more than one model was used in the run) to the terminal `result` event's `usage` object in `--output-format json`, mirroring what OTel's `chat` spans already export. This would make `--output-format json` self-sufficient for accurate cost accounting without requiring a full OTel pipeline for simple scripting/automation use cases.

### Example prompts or workflows

1. CI pipeline running `copilot -p "" --output-format json` per job step, parsing `result.usage` directly to log real per-task USD cost - no OTel collector needed.
2. Agent-orchestration frameworks (e.g. workflow engines that shell out to `copilot` as one of several interchangeable LLM-CLI backends) recording accurate per-node cost alongside output/session data from a single JSON parse.
3. Budget-alerting scripts that tail `result` events and sum `costUSD`/`aiCredits` across many non-interactive invocations without standing up OTel infrastructure just to get numbers already in memory.
4. Local dev tooling that shows "this command cost $X" right after a scripted `-p` call, matching what the interactive footer already shows for interactive sessions.

### Additional context

Environment: GitHub Copilot CLI 1.0.70, Windows.

Raw evidence available on request (unfiltered JSONL dumps + OTel raw export files from side-by-side runs of the same prompt with/without OTel enabled, plus the interactive footer output used for cross-validation). Happy to attach as files once this issue is reviewed.

Root-cause hypothesis: `result.usage` looks like a schema left over from the pre-"AI Credits" (legacy premium-request) billing era that was never updated when AI Credits + full token/cost telemetry was added via OTel - i.e. a schema sync gap rather than an intentional interactive-only restriction (`copilot help billing` documents credit/token visibility only through interactive surfaces - footer, /statusline, /model, /context, /usage, /exit - but OTel proves the same data is available non-interactively too).

Hướng dẫn đóng góp

Mở hướng dẫn đóng góp

Hướng nghiên cứu

Start at the CLI's --output-format json terminal result event and compare its usage object with the usage fields exposed by the OpenTelemetry file exporter described in copilot help monitoring. Done means the JSON result includes input, output, cache, and AI-credit or cost data, including per-model breakdowns when applicable, while preserving existing usage fields.

Do mô hình lập chỉ mục viết ra từ nội dung của issue.

Đánh giá

Công nghệ
json
Lĩnh vực
cli, observability
Loại issue
Tính năng
Độ khó
4/5
Thời gian dự kiến
3-5 ngày
Mức độ hoạt động
Ít trao đổi
Độ rõ ràng
Khá rõ ràng
Mức phù hợp với người mới
55/100

Nhận issue mới trong hộp thư của bạn

Bản tóm tắt ngắn những issue GitHub phù hợp với người mới.