fix(llma): ingestion silently drops AI events with nested token objects
@carlos-marchal-ph is already working on this.
Since May 6, 2026.
- Dominant language
- Python
- Stars
- 39.9k
- Forks
- 3.4k
- Avg merge
- 6h 51m
- Merged PRs (30d)
- 232
Description
Problem
When an SDK sends $ai_input_tokens or $ai_output_tokens as nested objects (e.g. { total: 10585, noCache: 10585, cacheRead: 0 }) instead of plain numbers, the validateAiEventTokensStep in the ingestion pipeline silently drops the entire event.
This affects users on older versions of @posthog/ai (< 7.3.0) using the Vercel AI SDK with providers like Amazon Bedrock. The Vercel AI SDK V3 returns token usage as structured objects, and older SDK versions pass them through without flattening.
Root cause
validateAiEventTokensStep (nodejs/src/ingestion/event-preprocessing/validate-ai-event-tokens.ts) checks each token property with isValidBigDecimalInput(), which returns false for objects. This causes the event to be dropped via drop() with no user-visible feedback.
The SDK-side fix landed in @posthog/ai@7.3.0 (PR posthog/posthog-js#2711) which added extractTokenCount() to normalize V3 token objects to plain numbers before sending. But users on older SDK versions still hit this silently.
Proposed fix
Add a normalization step in validateAiEventTokensStep that extracts the total field from nested token objects before validation. If the value is an object with a numeric total property, replace the property value with that number and continue processing instead of dropping.
This is a defense-in-depth improvement — the SDK fix is the primary solution, but the ingestion pipeline should be resilient to this shape of data.
Workaround
Upgrade @posthog/ai to >= 7.3.0 (latest: 7.9.3).
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.