LLM.get_context_window_size() returns the wrong window for anthropic.claude-v2:1 (last matching prefix wins)
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 58.8k
- Forks
- 8.5k
- Avg merge
- 1d 15h
- Merged PRs (30d)
- 109
Description
Description
LLM.get_context_window_size() (lib/crewai/src/crewai/llm.py) iterates LLM_CONTEXT_WINDOW_SIZES and lets the last matching entry win — it never breaks, so whichever matching key comes last in dict order decides the value, not the most specific one.
The table contains both:
"anthropic.claude-v2:1": 200000, # Claude 2.1
"anthropic.claude-v2": 100000, # Claude 2.0 — comes after, and shadows it
Since "anthropic.claude-v2:1".startswith("anthropic.claude-v2") is true, a Bedrock Claude 2.1 model resolves through the Claude 2.0 entry and gets 100000 instead of the 200000 entry that exists for it (85000 vs 170000 usable at the current ratio). The longer, correct entry can never win for any model name it covers.
Steps to Reproduce
from crewai import LLM
llm = LLM(model="anthropic.claude-v2:1", is_litellm=True)
print(llm.get_context_window_size()) # 85000 (100000 * 0.85) — wrong
# expected: 170000 (200000 * 0.85)
Generalized (insertion order decides, independent of the real table):
from unittest.mock import patch
from crewai import LLM
with patch.dict("crewai.llm.LLM_CONTEXT_WINDOW_SIZES",
{"acme-model-large": 128000, "acme-model": 8192}, clear=True):
print(LLM(model="acme-model-large", is_litellm=True).get_context_window_size())
# 6963 (8192 * 0.85) — the shorter, later key wins
Expected behavior
The most specific (longest) matching prefix should win, regardless of dict insertion order. This also matters right now because #7303 proposes adding "o1" / "o3" entries: appending them at the end of the dict would newly shadow the existing "o1-mini" / "o1-preview" / "o3-mini" keys under the current last-match-wins loop.
Environment
- crewAI
main(894898f),lib/crewai - Python 3.13, Windows 11
- Verified with
LLM_CONTEXT_WINDOW_SIZESas committed; no LLM calls needed.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in lib/crewai/src/crewai/llm.py at LLM.get_context_window_size() and inspect LLM_CONTEXT_WINDOW_SIZES. Reproduce the issue with anthropic.claude-v2:1 and the provided patched dictionary, then add regression coverage showing that the longest matching prefix wins regardless of insertion order. Done means the Claude 2.1 lookup returns 170000 and the generalized case no longer selects the shorter prefix.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- ai, backend
- Issue type
- Bug
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 88/100