Gemini 2.0 Flash Thinking window is wrong (1M, not 32k) and Bedrock provider resolves Claude 2.1 to 100k
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 58.8k
- Forks
- 8.5k
- Avg merge
- 1d 15h
- Merged PRs (30d)
- 109
Description
Description
Two context-window values in the provider tables are wrong or unreachable:
- gemini-2.0-flash-thinking is a 1M-token model, not 32k. Both
LLM_CONTEXT_WINDOW_SIZES("gemini-2.0-flash-thinking-exp-01-21": 32768) and the Gemini native provider ("gemini-2.0-flash-thinking": 32768) carry the 32,768 figure, but that belonged to the Decemberexp-1219snapshot. Theexp-01-21refresh raised the window to 1,048,576 tokens (see Google's long-context documentation and coverage of the January upgrade). WithLLM(model="gemini-2.0-flash-thinking-exp-01-21", is_litellm=True)the litellm path currently returns 32768 × ratio. - The Gemini native provider's 32k entry is also unreachable.
"gemini-2.0-flash-thinking": 32768sits after"gemini-2.0-flash": 1048576in a first-matchstartswithloop, sogemini-2.0-flash-thinking-exp-01-21resolves through the plaingemini-2.0-flashprefix and returns 1M. The two paths disagree with each other. - The Bedrock provider table has no
anthropic.claude-v2:1entry, so Claude 2.1 resolves through theanthropic.claude-v2prefix (Claude 2.0) and returns 100k instead of the 200k window llm.py's shared table already assigns to the:1model id.
Steps to Reproduce
from crewai import LLM
# litellm path: returns 32768 * 0.85, expected 1048576 * 0.85
LLM(model="gemini-2.0-flash-thinking-exp-01-21", is_litellm=True).get_context_window_size()
# native gemini path: returns 1048576 * 0.85 (accidentally correct, table entry dead)
LLM(model="google/gemini-2.0-flash-thinking-exp-01-21").get_context_window_size()
# native bedrock path: returns 100000 * 0.85, expected 200000 * 0.85
LLM(model="bedrock/anthropic.claude-v2:1").get_context_window_size()
Expected behavior
All three paths return the model's official window. Related to #7436 / #7437 (the lookup-order issue in the shared table); this issue is about the stale values and the missing Bedrock entry.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the provider tables named in the issue: LLM_CONTEXT_WINDOW_SIZES in the shared llm.py path, the Gemini native provider table, and the Bedrock provider table. Run the three supplied LLM.get_context_window_size() reproductions to confirm the lookup results. Done means Gemini Thinking resolves to 1,048,576 in both paths and Bedrock anthropic.claude-v2:1 resolves to 200,000, with the reported ratios applied.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- ai
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 72/100