[BUG] Gemini gemini-2.0-flash-thinking returns the wrong context window (891289 instead of 27852)
A pull request for this has already been merged.
- #7003 by @a-yeyang — merged
- Dominant language
- Python
- Stars
- 58.8k
- Forks
- 8.5k
- Avg merge
- 1d 15h
- Merged PRs (30d)
- 109
Description
Description
In GeminiCompletion.get_context_window_size() the lookup table has "gemini-2.0-flash" listed before "gemini-2.0-flash-thinking", and the loop returns on the first match. So the thinking entry is never reached.
# lib/crewai/src/crewai/llms/providers/gemini/completion.py:1356
context_windows = {
"gemini-3-pro-preview": 1048576,
"gemini-2.0-flash": 1048576, # matches first
"gemini-2.0-flash-thinking": 32768, # never reached
"gemini-2.0-flash-lite": 1048576,
...
}
for model_prefix, size in context_windows.items():
if self.model.startswith(model_prefix):
return int(size * CONTEXT_WINDOW_USAGE_RATIO)
"gemini-2.0-flash-thinking".startswith("gemini-2.0-flash") is True, so the 32768 line is dead code. It has been there since the file was added in v1.0.0 (#3618).
So the model reports a context window of 891289 when the real one is 32768 (27852 after the ratio).
I am not sure how much this affects users in practice, since gemini-2.0-flash-thinking-exp is a preview model. But the value returned is wrong, and the table entry that has the right value can never run.
Steps to Reproduce
- Install crewai 1.15.17
- Run the code below. No API key or network needed, it is only a table lookup.
- Compare the output with
int(32768 * 0.85)= 27852
Expected behavior
gemini-2.0-flash-thinking-* should return 27852, which is the value the table already has for it.
In general the lookup should pick the longest matching prefix, so a specific key is not hidden by a shorter key placed above it.
Screenshots/Code snippets
from crewai import LLM
llm = LLM(model="gemini/gemini-2.0-flash-thinking-exp-01-21")
print(llm.get_context_window_size())
Actual:
891289 # this is int(1048576 * 0.85), the gemini-2.0-flash value
Expected:
27852 # int(32768 * 0.85)
One more thing: the same model gives the correct 27852 through crewai/llm.py, because that loop assigns instead of returning and ends on the last match. So the two paths return different values for the same model id.
Additional context
The OpenAI and Azure providers already handle this. They have this comment:
# lib/crewai/src/crewai/llms/providers/openai/completion.py
# Longest prefix first. Always insert new keys in that order so
# startswith prefers gpt-5.6 over gpt-5, gpt-4o-mini over gpt-4o, etc.
The Gemini table does not follow that order. Two ways to fix it:
- Move
"gemini-2.0-flash-thinking"above"gemini-2.0-flash", same as the other providers do. - Pick the longest matching prefix instead of the first one, so the order does not matter. I would prefer this one, because the table gets edited fairly often (#7012, #7125) and the same mistake can happen again.
I checked the other prefix tables under lib/ with a small script and this is the only entry left whose value cannot be reached.
Happy to send a PR with a regression test if you tell me which of the two fixes you prefer.
Disclosure: I used an AI agent (Claude Code) to help find and check this. I reproduced it locally before filing.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in lib/crewai/src/crewai/llms/providers/gemini/completion.py at GeminiCompletion.get_context_window_size() and compare the matching behavior with crewai/llm.py and the OpenAI provider’s prefix lookup. Run the provided no-network reproduction with the Gemini model identifier. Done means the thinking model returns 27852 and both lookup paths agree, with a regression test covering the prefix collision.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- ai, backend
- Issue type
- Bug
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Stale
- Clarity
- Clearly specified
- Newbie friendliness
- 45/100