crewAIInc / crewAIInc/crewAI

[BUG] LiteLLM streaming corrupts reassembled tool calls: dropped ids and doubled arguments

Open
#7,534 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
58.8k
Forks
8.5k
Avg merge
1d 15h
Merged PRs (30d)
109

Description

Description

When LLM.stream=True on the LiteLLM fallback path (LLM(..., is_litellm=True), i.e. any model not routed to a native SDK provider), streamed tool_calls deltas are accumulated per index and rebuilt into new ChatCompletionDeltaToolCall objects. The reassembly is wrong twice:

  1. The wire id is never copied. Every rebuilt call returns id=None. The agent executor extracts that id (extract_tool_call_infogetattr(tool_call, "id", ...)) and writes the assistant turn as tool_calls[].id = null plus tool_call_id = null on each role="tool" result. The next LLM request then carries null ids — providers that require string ids (e.g. OpenAI: messages[].tool_calls[].id, messages[].tool_call_id) reject it, and tool results can no longer be correlated to their calls.
  2. The async path double-appends the arguments. After the stream ends, _ahandle_streaming_response replays the rebuilt calls into _handle_streaming_tool_calls passing the same accumulated_tool_args dict that already holds the complete arguments. The handler does accumulator.function.arguments += tool_call.function.arguments, producing {"a":1}{"a":1}, so json.loads always fails and the tool call is silently dropped — acall returns the accumulated text ("") instead of executing the function whenever available_functions is provided.
Steps to Reproduce
llm = LLM(model="gpt-4o-mini", is_litellm=True, stream=True)

# stream chunks: first delta carries id + name + half the args,
# second delta carries the rest of the args (litellm.types.utils objects)
result = llm.call("weather?", tools=[get_weather_schema])
assert result[0].id == "call_abc123"   # actual: None

result = await llm.acall(
    "weather?", tools=[get_weather_schema],
    available_functions={"get_weather": fn},
)
# actual: fn is never called and result == ""

The regression tests in lib/crewai/tests/llms/litellm/test_litellm_streaming_tool_calls.py reproduce both failures against litellm.completion / litellm.acompletion mocked with ModelResponseStream chunks — no network needed.

Expected behavior

Each rebuilt tool call keeps the id the provider streamed (e.g. call_abc123), and the async streaming path executes the tool from available_functions with the original, parseable arguments.

Screenshots/Code snippets

Affected code:

  • lib/crewai/src/crewai/llm.py
    • AccumulatedToolArgs (~line 367): no id field.
    • LLM._handle_streaming_response (~line 1039): rebuilds ChatCompletionDeltaToolCall(index=..., function=...) without id.
    • LLM._handle_streaming_tool_calls (~line 1159): accumulates function.name/function.arguments only — tool_call.id is discarded.
    • LLM._ahandle_streaming_response (~lines 1648–1731): inline accumulation also discards tool_call.id, and the post-stream replay reuses the populated accumulated_tool_args, doubling the arguments.
Operating System

Ubuntu 22.04

Python Version

3.12

crewAI Version

1.15.22 (main @ 5c33fe4)

crewAI Tools Version

1.15.22

Virtual Environment

Venv

Evidence
  • Crew(stream=True) / LLM(stream=True) with a LiteLLM-routed model: the first tool call returns to the executor with id=None, so the follow-up request contains tool_calls[].id = null / tool_call_id = null and fails provider-side validation (or silently breaks call/result correlation).
  • await llm.acall(..., tools=..., available_functions=...) with streaming enabled: the tool is never executed and the call returns "".

Pre-fix, the regression tests fail with id=None (sync and async) and with the tool function never called (async with available_functions).

Possible Solution
  • Add id: str | None = None to AccumulatedToolArgs and capture tool_call.id at both accumulation sites (_handle_streaming_tool_calls and the async inline loop).
  • Set id=tool_arg.id when rebuilding ChatCompletionDeltaToolCall in both _handle_streaming_response and _ahandle_streaming_response.
  • In _ahandle_streaming_response, replay the rebuilt calls against a fresh defaultdict(AccumulatedToolArgs) instead of the populated accumulator, so the arguments stay parseable and the tool executes.
Additional context

Found and written with an AI coding agent (Devin, reviewed by Claude Code); please apply the llm-generated label. A fix with offline regression tests is ready and will be linked here.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with lib/crewai/src/crewai/llm.py, especially AccumulatedToolArgs and the sync and async streaming handlers around the cited lines. Run lib/crewai/tests/llms/litellm/test_litellm_streaming_tool_calls.py first to observe the mocked streaming failures. Done means rebuilt calls preserve provider ids and the async available_functions path executes the tool with parseable arguments.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
api, backend, testing-qa
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
78/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.