crewAIInc / crewAIInc/crewAI

[BUG] Bedrock cross-region inference profiles get the 8192-token fallback context window

Open
#6,244 4 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

no-issue-activity
Dominant language
Python
Stars
58.8k
Forks
8.5k
Avg merge
1d 15h
Merged PRs (30d)
109

Description

First, thank you to the crewAI maintainers for the native Bedrock provider — the Converse-API integration is genuinely nice to build on. I think I've found a small gap in how the context window is sized for cross-region inference profiles, and I have a fix + tests ready.

Description

AWS recommends invoking newer Bedrock models (Claude 4, etc.) through cross-region inference profiles, whose model ids carry a geographic prefix: us., eu., apac., us-gov. — e.g. bedrock/us.anthropic.claude-sonnet-4-20250514-v1:0.

BedrockCompletion.get_context_window_size() looks the model up in a table keyed on the bare id (anthropic.claude-sonnet-4) using self.model.startswith(model_prefix). When the model carries a region prefix, none of the keys match, so it falls through to the 8192 default. A 200K-token model is therefore treated as ~8K, which triggers premature summarization/truncation of the conversation.

Reproduction
from crewai import LLM

base    = LLM(model="bedrock/anthropic.claude-sonnet-4-20250514-v1:0")
profile = LLM(model="bedrock/us.anthropic.claude-sonnet-4-20250514-v1:0")  # AWS-recommended form

print(base.get_context_window_size())     # 170000  (200000 * 0.85 ratio)
print(profile.get_context_window_size())  # 6963    (8192   * 0.85)  <-- wrong
Model id Expected Actual (before fix)
anthropic.claude-sonnet-4-... (bare) 170000 170000 ✅
us.anthropic.claude-sonnet-4-... 170000 6963 ❌
eu.anthropic.claude-sonnet-4-... 170000 6963 ❌
apac.anthropic.claude-sonnet-4-... 170000 6963 ❌

The same prefix issue affects supports_multimodal(): it hardcodes only us. variants, so eu./apac. Claude/Nova profiles are reported as non-vision.

Root cause

The lookups key on the bare model id but compare against self.model, which (correctly) keeps the cross-region prefix. The sibling validator _normalize_bedrock_fields is already prefix-agnostic — this is one spot where the prefix wasn't accounted for.

Proposed fix

Strip the cross-region prefix before the table lookup via a small shared helper, and reuse it in supports_multimodal. PR linked below.

Possibly related real-world report: #3791 (cross-region inference breakage). Happy to adjust scope if maintainers prefer.


This issue was prepared with the help of an AI agent (Claude Code); a human reviewed the analysis, reproduction, and proposed fix before submitting.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start at BedrockCompletion.get_context_window_size() and supports_multimodal(), then compare their model-id matching with the prefix-agnostic _normalize_bedrock_fields validator. Run or extend the existing Bedrock tests to cover bare, us., eu., and apac. model IDs; done means context sizes and multimodal support are consistent across those forms.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, python
Domain
ai, cloud
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
72/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.