crewAIInc / crewAIInc/crewAI

LLM.get_context_window_size() returns the wrong window for anthropic.claude-v2:1 (last matching prefix wins)

Open Beginner friendly
#7,436 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
58.8k
Forks
8.5k
Avg merge
1d 15h
Merged PRs (30d)
109

Description

Description

LLM.get_context_window_size() (lib/crewai/src/crewai/llm.py) iterates LLM_CONTEXT_WINDOW_SIZES and lets the last matching entry win — it never breaks, so whichever matching key comes last in dict order decides the value, not the most specific one.

The table contains both:

"anthropic.claude-v2:1": 200000,   # Claude 2.1
"anthropic.claude-v2": 100000,     # Claude 2.0 — comes after, and shadows it

Since "anthropic.claude-v2:1".startswith("anthropic.claude-v2") is true, a Bedrock Claude 2.1 model resolves through the Claude 2.0 entry and gets 100000 instead of the 200000 entry that exists for it (85000 vs 170000 usable at the current ratio). The longer, correct entry can never win for any model name it covers.

Steps to Reproduce
from crewai import LLM

llm = LLM(model="anthropic.claude-v2:1", is_litellm=True)
print(llm.get_context_window_size())   # 85000 (100000 * 0.85) — wrong
# expected: 170000 (200000 * 0.85)

Generalized (insertion order decides, independent of the real table):

from unittest.mock import patch
from crewai import LLM

with patch.dict("crewai.llm.LLM_CONTEXT_WINDOW_SIZES",
                {"acme-model-large": 128000, "acme-model": 8192}, clear=True):
    print(LLM(model="acme-model-large", is_litellm=True).get_context_window_size())
    # 6963 (8192 * 0.85) — the shorter, later key wins
Expected behavior

The most specific (longest) matching prefix should win, regardless of dict insertion order. This also matters right now because #7303 proposes adding "o1" / "o3" entries: appending them at the end of the dict would newly shadow the existing "o1-mini" / "o1-preview" / "o3-mini" keys under the current last-match-wins loop.

Environment
  • crewAI main (894898f), lib/crewai
  • Python 3.13, Windows 11
  • Verified with LLM_CONTEXT_WINDOW_SIZES as committed; no LLM calls needed.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in lib/crewai/src/crewai/llm.py at LLM.get_context_window_size() and inspect LLM_CONTEXT_WINDOW_SIZES. Reproduce the issue with anthropic.claude-v2:1 and the provided patched dictionary, then add regression coverage showing that the longest matching prefix wins regardless of insertion order. Done means the Claude 2.1 lookup returns 170000 and the generalized case no longer selects the shorter prefix.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
ai, backend
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
88/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.