cloudflare / cloudflare/cloudflare-docs

Workers AI: gemma-4-26b-a4b-it context window is 131072 (128K), not 256K as documented

Open
#29,731 1 comment 0 reactions 7 assignees Claimed by @zeke View on GitHub
content:edit documentation product:changelog product:workers-ai stale
Dominant language
MDX
Stars
5.2k
Forks
16.7k
Avg merge
2d 6h
Merged PRs (30d)
337

Description

### Existing documentation URL(s)

- https://developers.cloudflare.com/workers-ai/models/gemma-4-26b-a4b-it/
- - https://developers.cloudflare.com/changelog/post/2026-04-04-gemma-4-26b-a4b-workers-ai/

### What changes are you suggesting?

## Problem

The documentation for `@cf/google/gemma-4-26b-a4b-it` states a **256,000 token context window**, but the actual API deployment returns a **131,072 token (128K) limit**.

When sending requests that exceed 131K total tokens (input + output), the API returns a 400 error with this message:

```
AiError: This model's maximum context length is 131072 tokens.
However, you requested 16000 output tokens and your prompt contains
at least 115073 input tokens, for a total of at least 131073 tokens.
Please reduce the length of the input prompt or reduce max_tokens.
```

## Where the discrepancy appears

- **Model page** says "Context Window: 256,000 tokens"
- - **Changelog** says "256K context window"
- - - **Actual API response** enforces 131,072 tokens
## Expected outcome

Either:
1. **Update the documentation** to reflect the actual 131,072 token limit, OR
2. 2. **Update the vLLM deployment** to use `--max-model-len 262144` (256K) as advertised — Google's official model card confirms the 26B A4B variant supports 256K context
## Reproduction

Call the `/v1/chat/completions` endpoint with `@cf/google/gemma-4-26b-a4b-it` and a prompt exceeding ~115K tokens + 16K max_tokens. The API rejects with the 131072 limit error instead of allowing up to 256K.

## Impact

This affects production applications that rely on the documented 256K context window for long conversations, legal documents, and RAG pipelines. Approximately 27% of our requests exceed 131K tokens but are well within the documented 256K limit.

### Additional information

_No response_

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.