[BUG] Bedrock cross-region inference profiles get the 8192-token fallback context window
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 58.8k
- Forks
- 8.5k
- Avg merge
- 1d 15h
- Merged PRs (30d)
- 109
Description
First, thank you to the crewAI maintainers for the native Bedrock provider — the Converse-API integration is genuinely nice to build on. I think I've found a small gap in how the context window is sized for cross-region inference profiles, and I have a fix + tests ready.
Description
AWS recommends invoking newer Bedrock models (Claude 4, etc.) through cross-region inference profiles, whose model ids carry a geographic prefix: us., eu., apac., us-gov. — e.g. bedrock/us.anthropic.claude-sonnet-4-20250514-v1:0.
BedrockCompletion.get_context_window_size() looks the model up in a table keyed on the bare id (anthropic.claude-sonnet-4) using self.model.startswith(model_prefix). When the model carries a region prefix, none of the keys match, so it falls through to the 8192 default. A 200K-token model is therefore treated as ~8K, which triggers premature summarization/truncation of the conversation.
Reproduction
from crewai import LLM
base = LLM(model="bedrock/anthropic.claude-sonnet-4-20250514-v1:0")
profile = LLM(model="bedrock/us.anthropic.claude-sonnet-4-20250514-v1:0") # AWS-recommended form
print(base.get_context_window_size()) # 170000 (200000 * 0.85 ratio)
print(profile.get_context_window_size()) # 6963 (8192 * 0.85) <-- wrong
| Model id | Expected | Actual (before fix) |
|---|---|---|
anthropic.claude-sonnet-4-... (bare) |
170000 | 170000 ✅ |
us.anthropic.claude-sonnet-4-... |
170000 | 6963 ❌ |
eu.anthropic.claude-sonnet-4-... |
170000 | 6963 ❌ |
apac.anthropic.claude-sonnet-4-... |
170000 | 6963 ❌ |
The same prefix issue affects supports_multimodal(): it hardcodes only us. variants, so eu./apac. Claude/Nova profiles are reported as non-vision.
Root cause
The lookups key on the bare model id but compare against self.model, which (correctly) keeps the cross-region prefix. The sibling validator _normalize_bedrock_fields is already prefix-agnostic — this is one spot where the prefix wasn't accounted for.
Proposed fix
Strip the cross-region prefix before the table lookup via a small shared helper, and reuse it in supports_multimodal. PR linked below.
Possibly related real-world report: #3791 (cross-region inference breakage). Happy to adjust scope if maintainers prefer.
This issue was prepared with the help of an AI agent (Claude Code); a human reviewed the analysis, reproduction, and proposed fix before submitting.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start at BedrockCompletion.get_context_window_size() and supports_multimodal(), then compare their model-id matching with the prefix-agnostic _normalize_bedrock_fields validator. Run or extend the existing Bedrock tests to cover bare, us., eu., and apac. model IDs; done means context sizes and multimodal support are consistent across those forms.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, python
- Domain
- ai, cloud
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 72/100