getsentry / getsentry/sentry-javascript

Vercel AI: reasoning tokens never reach `invoke_agent` parents, and streamed parents report candidate-only output

Offen
#23,993 1 Kommentar 0 Reaktionen 1 zugewiesene Person Beansprucht von @RulaKhaled Auf GitHub ansehen
javascript
Vorherrschende Sprache
TypeScript
Sterne
8.7k
Forks
1.8k
Ø Merge
1 T. 17 Std.
Gemergte PRs (30 T.)
523

Beschreibung

Follow-up to #23433, which fixes Gemini reasoning tokens on model-call (`gen_ai.generate_content`) spans. Two gaps are deliberately out of scope there and need a decision.

### Background

Per the [conventions](https://getsentry.github.io/sentry-conventions/attributes/gen_ai/), `gen_ai.usage.output_tokens` is reasoning-inclusive and `gen_ai.usage.reasoning.output_tokens` is a subset of it. On `ai` v4/v5 with `@ai-sdk/google`, the SDK's `outputTokens` is candidate-only — reasoning ("thoughts") is exposed only through `providerMetadata.google.usageMetadata`. #23433 recomputes output on model-call spans from `candidatesTokenCount + thoughtsTokenCount`.

It cannot do the same on `gen_ai.invoke_agent` parents: their `ai.usage.*` is `totalUsage` summed across every step, while `ai.response.providerMetadata` describes the **last step only**. So #23433 gates `output_tokens`, `total_tokens` and `reasoning.output_tokens` off parents entirely.

### Gap 1 — streamed parents report candidate-only output

With `spanStreamingIntegration` enabled, each span is emitted independently, so there is no pass that can re-derive a parent from its children. Measured on v10 with #23433 applied:

| parent | static (event processor) | streamed |
|---|---|---|
| single-step | `output 101, total 115` ✅ | `output 1, total 15` ⚠️ |
| multi-step | `output 201, total 1101` ✅ | `output 21, total 921` ⚠️ |

The static path is correct because `applyAccumulatedTokens` overwrites the parent by summing its (already corrected) children. The streamed path has no equivalent.

### Gap 2 — `reasoning.output_tokens` never reaches parents

`accumulateTokensForParent` / `applyAccumulatedTokens` sum input and output but never reasoning, so unlike output/total there is no fallback once it is gated. A parent therefore carries no reasoning count on either path. Writing the ungated value would be wrong rather than merely absent: on a two-step call it reports the last step's reasoning as the whole call's (e.g. `100` where the true aggregate is `80 + 100 = 180`).

### Options to investigate

1. **Sum reasoning in `accumulateTokensForParent` / `applyAccumulatedTokens`.** Gives parents a true aggregate reasoning count and closes Gap 2 on the static path. Does not help the streamed path, which has no accumulation. Touches shared token plumbing that every provider flows through, so it needs its own testing.

2. **Read the SDK's own reasoning tokens instead of deriving them from `providerMetadata`.** `ai` v6+ exposes `ai.usage.outputTokenDetails.reasoningTokens` (and the deprecated `ai.usage.reasoningTokens`) directly on the span, sourced from `usage.outputTokens.reasoning`, and v6 `outputTokens` is already `total` = text + reasoning. Nothing in the SDK reads either today. This is provider-agnostic, works on both paths, and would cover Gemini, OpenAI and Anthropic at once — but only for SDK versions that emit it.

### Also worth folding in

- On `v10`, the OpenAI branch of `getProviderMetadataAttributes` writes `gen_ai.usage.output_tokens.reasoning`, which `@sentry/conventions` marks deprecated in favour of `gen_ai.usage.reasoning.output_tokens`. It has the same last-step-only issue on parents.
- On `develop`, the Vercel AI OTel path was removed in #23384. `enrichSpanOnEnd` in `vercel-ai-dc-subscriber.ts` reads input/output/total but never `outputTokenDetails.reasoningTokens`, so `gen_ai.usage.reasoning.output_tokens` is currently only ever populated for OpenAI. Option 2 likely applies there more cleanly than on `v10`.

Beitragsleitfaden

Beitragsleitfaden öffnen

Bewertung

Dieses Issue wurde noch nicht bewertet.

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.