Analytics not functioning properly when Accept-Encoding header is set in request
Open
Nobody has claimed this yet.
Area/Management
Aspect/API
Severity/Blocker
Type/Bug
- Dominant language
- Go
- Stars
- 71
- Forks
- 111
- Avg merge
- 1d 14h
- Merged PRs (30d)
- 110
Description
Please select the area the issue is related to
Area/Management (Management API or Management Portal UI)
Please select the aspect the issue is related to
Aspect/API (API backends, definitions, contracts, interfaces, OpenAPI)
Description
Both the OpenAI and Anthropic Python and Node.js SDKs automatically include the Accept-Encoding header by default. This causes the response to be compressed, in it's current implementation token analytics are not tracked when the response is in this format.
Steps to Reproduce
- Create a self hosted ai-gateway
- Create an LLM provider using anthropic (any provider works)
- Deploy LLM provider
- Make a call to the LLM provider using postman (notice that the
Accept-Encodingheader is sent by default) - Go to the insights page on the ai workspace
- Notice that the Completion / Prompt / Total tokens are not being tracked
- Resend the request with the
Accept-Encodingheader removed - Notice that the Completion / Prompt / Total tokens are now being tracked
Severity Level of the Issue
Severity/Blocker (Core functionality is broken and consumer is blocked. Need immediate attention)
Environment Details (with versions)
Production, ai-gateway: v0.9.0
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.