LLM analytics: cost double-counts Gemini reasoning tokens (output tokens already inclusive)
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 39.9k
- Forks
- 3.4k
- Avg merge
- 6h 51m
- Merged PRs (30d)
- 232
Description
Bug
LLM analytics cost calculation prices Gemini reasoning tokens twice: $ai_output_tokens from the Vercel AI SDK already includes reasoning tokens, but the cost pipeline adds $ai_reasoning_tokens on top, inflating $ai_total_cost_usd.
Repro
gemini-2.5-flash via Vercel AI SDK v6, wrapped with @posthog/ai@8.3.0 withTracing (Node, posthog-node@5.36.2).
Provider usage returned by the SDK:
{"inputTokens":9,"outputTokens":22,"outputTokenDetails":{"textTokens":1,"reasoningTokens":21},"totalTokens":31,"raw":{"thoughtsTokenCount":21,"promptTokenCount":9,"candidatesTokenCount":1,"totalTokenCount":31}}
Event captured with $ai_input_tokens: 9, $ai_output_tokens: 22 (inclusive of the 21 reasoning tokens — matches Google's semantics, where candidatesTokenCount + thoughtsTokenCount are both billed as output), $ai_reasoning_tokens: 21.
Expected
At Gemini list prices ($0.30/M input, $2.50/M output incl. thinking):
9 × 0.30/1e6 + 22 × 2.50/1e6 = $0.0000577
Actual
$ai_total_cost_usd = 0.0001102
which is exactly 9 × 0.30/1e6 + (22 + 21) × 2.50/1e6 — the 21 reasoning tokens are priced a second time on top of the already-inclusive output count.
Control: a zero-reasoning call on the same stack (gemini-2.5-flash-lite, LangChain callback handler, 9 in / 1 out) prices exactly right at $0.0000013, isolating the discrepancy to reasoning-token handling.
Suggested fix
Same class of problem as inclusive/exclusive cache-token reporting, which already has auto-detection plus a $ai_cache_reporting_exclusive override. Reasoning tokens need the equivalent: auto-detect providers that report output inclusive of reasoning (Gemini / Vercel AI SDK), or support a $ai_reasoning_reporting_exclusive property.
Workaround we're using meanwhile: pre-calculated $ai_input_cost_usd / $ai_output_cost_usd via posthogProperties.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the LLM analytics cost pipeline and the Gemini/Vercel AI SDK usage mapping described in the reproduction. Compare how inclusive reasoning and output tokens are represented, then verify that the reported total cost prices the reasoning tokens only once while preserving the zero-reasoning control case.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- node.js
- Domain
- ai, analytics
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100