getsentry / getsentry/sentry-javascript

Vercel AI: reasoning tokens never reach `invoke_agent` parents, and streamed parents report candidate-only output

Abierto
#23,993 1 comentario 0 reacciones 1 asignado Reclamado por @RulaKhaled Ver en GitHub
javascript
Lenguaje dominante
TypeScript
Estrellas
8.7k
Forks
1.8k
Merge medio
1 d 17 h
PR fusionados (30 d)
515

Descripción

Follow-up to #23433, which fixes Gemini reasoning tokens on model-call (`gen_ai.generate_content`) spans. Two gaps are deliberately out of scope there and need a decision.

### Background

Per the [conventions](https://getsentry.github.io/sentry-conventions/attributes/gen_ai/), `gen_ai.usage.output_tokens` is reasoning-inclusive and `gen_ai.usage.reasoning.output_tokens` is a subset of it. On `ai` v4/v5 with `@ai-sdk/google`, the SDK's `outputTokens` is candidate-only — reasoning ("thoughts") is exposed only through `providerMetadata.google.usageMetadata`. #23433 recomputes output on model-call spans from `candidatesTokenCount + thoughtsTokenCount`.

It cannot do the same on `gen_ai.invoke_agent` parents: their `ai.usage.*` is `totalUsage` summed across every step, while `ai.response.providerMetadata` describes the **last step only**. So #23433 gates `output_tokens`, `total_tokens` and `reasoning.output_tokens` off parents entirely.

### Gap 1 — streamed parents report candidate-only output

With `spanStreamingIntegration` enabled, each span is emitted independently, so there is no pass that can re-derive a parent from its children. Measured on v10 with #23433 applied:

| parent | static (event processor) | streamed |
|---|---|---|
| single-step | `output 101, total 115` ✅ | `output 1, total 15` ⚠️ |
| multi-step | `output 201, total 1101` ✅ | `output 21, total 921` ⚠️ |

The static path is correct because `applyAccumulatedTokens` overwrites the parent by summing its (already corrected) children. The streamed path has no equivalent.

### Gap 2 — `reasoning.output_tokens` never reaches parents

`accumulateTokensForParent` / `applyAccumulatedTokens` sum input and output but never reasoning, so unlike output/total there is no fallback once it is gated. A parent therefore carries no reasoning count on either path. Writing the ungated value would be wrong rather than merely absent: on a two-step call it reports the last step's reasoning as the whole call's (e.g. `100` where the true aggregate is `80 + 100 = 180`).

### Options to investigate

1. **Sum reasoning in `accumulateTokensForParent` / `applyAccumulatedTokens`.** Gives parents a true aggregate reasoning count and closes Gap 2 on the static path. Does not help the streamed path, which has no accumulation. Touches shared token plumbing that every provider flows through, so it needs its own testing.

2. **Read the SDK's own reasoning tokens instead of deriving them from `providerMetadata`.** `ai` v6+ exposes `ai.usage.outputTokenDetails.reasoningTokens` (and the deprecated `ai.usage.reasoningTokens`) directly on the span, sourced from `usage.outputTokens.reasoning`, and v6 `outputTokens` is already `total` = text + reasoning. Nothing in the SDK reads either today. This is provider-agnostic, works on both paths, and would cover Gemini, OpenAI and Anthropic at once — but only for SDK versions that emit it.

### Also worth folding in

- On `v10`, the OpenAI branch of `getProviderMetadataAttributes` writes `gen_ai.usage.output_tokens.reasoning`, which `@sentry/conventions` marks deprecated in favour of `gen_ai.usage.reasoning.output_tokens`. It has the same last-step-only issue on parents.
- On `develop`, the Vercel AI OTel path was removed in #23384. `enrichSpanOnEnd` in `vercel-ai-dc-subscriber.ts` reads input/output/total but never `outputTokenDetails.reasoningTokens`, so `gen_ai.usage.reasoning.output_tokens` is currently only ever populated for OpenAI. Option 2 likely applies there more cleanly than on `v10`.

Guía de contribución

Abrir la guía de contribución

Evaluación

Este issue todavía no se ha evaluado.

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.