LLM analytics trace totalCost excludes $ai_span $ai_total_cost_usd
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 39.9k
- Forks
- 3.4k
- Avg merge
- 6h 51m
- Merged PRs (30d)
- 232
Description
Summary
LLM Analytics trace-level totalCost currently appears to exclude $ai_total_cost_usd from $ai_span events.
In our traces, custom operations (for example reranking) are captured as $ai_span with valid $ai_total_cost_usd. The span-level UI shows the cost, but the trace header/table totalCost does not include it.
Expected behavior
Trace/session totalCost should include all AI event costs that belong to the trace, including $ai_span events with $ai_total_cost_usd.
Actual behavior
Trace/session totalCost includes only $ai_generation and $ai_embedding costs.
Evidence in code
Current trace queries sum cost only for event IN ('$ai_generation', '$ai_embedding'):
posthog/hogql_queries/ai/trace_query_runner.pyposthog/hogql_queries/ai/traces_query_runner.py
Both queries compute total_cost using:
sumIf(toFloat(properties.$ai_total_cost_usd), event IN ('$ai_generation', '$ai_embedding'))
Repro
- Send one
$ai_generationevent with$ai_total_cost_usd = 0.058 - Send one
$ai_spanevent in the same trace with$ai_total_cost_usd = 0.004 - Open trace in LLM Analytics
Result:
- Span shows
0.004correctly - Trace
totalCostshows0.058instead of0.062
Notes
This makes trace/session totals under-report costs whenever teams put billable work on $ai_span (rerank/search/tool/API operations) instead of $ai_generation/$ai_embedding.
If this is intentional, could docs explicitly state that trace/session totals only aggregate generation/embedding events?
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with posthog/hogql_queries/ai/trace_query_runner.py and posthog/hogql_queries/ai/traces_query_runner.py, focusing on the total_cost expressions and their event filters. Verify the trace and session totals against the repro values, including the $ai_span cost, and ensure both queries report the expected aggregate.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, sql
- Domain
- ai, backend
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Clearly specified
- Newbie friendliness
- 58/100