cloudflare / cloudflare/cloudflare-docs
Workers AI: gemma-4-26b-a4b-it context window is 131072 (128K), not 256K as documented
- Dominant language
- MDX
- Stars
- 5.2k
- Forks
- 16.7k
- Avg merge
- 2d 6h
- Merged PRs (30d)
- 337
Description
### Existing documentation URL(s)
- https://developers.cloudflare.com/workers-ai/models/gemma-4-26b-a4b-it/
- - https://developers.cloudflare.com/changelog/post/2026-04-04-gemma-4-26b-a4b-workers-ai/
### What changes are you suggesting?
## Problem
The documentation for `@cf/google/gemma-4-26b-a4b-it` states a **256,000 token context window**, but the actual API deployment returns a **131,072 token (128K) limit**.
When sending requests that exceed 131K total tokens (input + output), the API returns a 400 error with this message:
```
AiError: This model's maximum context length is 131072 tokens.
However, you requested 16000 output tokens and your prompt contains
at least 115073 input tokens, for a total of at least 131073 tokens.
Please reduce the length of the input prompt or reduce max_tokens.
```
## Where the discrepancy appears
- **Model page** says "Context Window: 256,000 tokens"
- - **Changelog** says "256K context window"
- - - **Actual API response** enforces 131,072 tokens
## Expected outcome
Either:
1. **Update the documentation** to reflect the actual 131,072 token limit, OR
2. 2. **Update the vLLM deployment** to use `--max-model-len 262144` (256K) as advertised — Google's official model card confirms the 26B A4B variant supports 256K context
## Reproduction
Call the `/v1/chat/completions` endpoint with `@cf/google/gemma-4-26b-a4b-it` and a prompt exceeding ~115K tokens + 16K max_tokens. The API rejects with the 131072 limit error instead of allowing up to 256K.
## Impact
This affects production applications that rely on the documented 256K context window for long conversations, legal documents, and RAG pipelines. Approximately 27% of our requests exceed 131K tokens but are well within the documented 256K limit.
### Additional information
_No response_
Contributor guide
Assessment
This issue has not been assessed yet.