[BUG] LiteLLM streaming corrupts reassembled tool calls: dropped ids and doubled arguments
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 58.8k
- Forks
- 8.5k
- Avg merge
- 1d 15h
- Merged PRs (30d)
- 109
Description
Description
When LLM.stream=True on the LiteLLM fallback path (LLM(..., is_litellm=True), i.e. any model not routed to a native SDK provider), streamed tool_calls deltas are accumulated per index and rebuilt into new ChatCompletionDeltaToolCall objects. The reassembly is wrong twice:
- The wire
idis never copied. Every rebuilt call returnsid=None. The agent executor extracts that id (extract_tool_call_info→getattr(tool_call, "id", ...)) and writes the assistant turn astool_calls[].id = nullplustool_call_id = nullon eachrole="tool"result. The next LLM request then carries null ids — providers that require string ids (e.g. OpenAI:messages[].tool_calls[].id,messages[].tool_call_id) reject it, and tool results can no longer be correlated to their calls. - The async path double-appends the arguments. After the stream ends,
_ahandle_streaming_responsereplays the rebuilt calls into_handle_streaming_tool_callspassing the sameaccumulated_tool_argsdict that already holds the complete arguments. The handler doesaccumulator.function.arguments += tool_call.function.arguments, producing{"a":1}{"a":1}, sojson.loadsalways fails and the tool call is silently dropped —acallreturns the accumulated text ("") instead of executing the function wheneveravailable_functionsis provided.
Steps to Reproduce
llm = LLM(model="gpt-4o-mini", is_litellm=True, stream=True)
# stream chunks: first delta carries id + name + half the args,
# second delta carries the rest of the args (litellm.types.utils objects)
result = llm.call("weather?", tools=[get_weather_schema])
assert result[0].id == "call_abc123" # actual: None
result = await llm.acall(
"weather?", tools=[get_weather_schema],
available_functions={"get_weather": fn},
)
# actual: fn is never called and result == ""
The regression tests in lib/crewai/tests/llms/litellm/test_litellm_streaming_tool_calls.py reproduce both failures against litellm.completion / litellm.acompletion mocked with ModelResponseStream chunks — no network needed.
Expected behavior
Each rebuilt tool call keeps the id the provider streamed (e.g. call_abc123), and the async streaming path executes the tool from available_functions with the original, parseable arguments.
Screenshots/Code snippets
Affected code:
lib/crewai/src/crewai/llm.pyAccumulatedToolArgs(~line 367): noidfield.LLM._handle_streaming_response(~line 1039): rebuildsChatCompletionDeltaToolCall(index=..., function=...)withoutid.LLM._handle_streaming_tool_calls(~line 1159): accumulatesfunction.name/function.argumentsonly —tool_call.idis discarded.LLM._ahandle_streaming_response(~lines 1648–1731): inline accumulation also discardstool_call.id, and the post-stream replay reuses the populatedaccumulated_tool_args, doubling the arguments.
Operating System
Ubuntu 22.04
Python Version
3.12
crewAI Version
1.15.22 (main @ 5c33fe4)
crewAI Tools Version
1.15.22
Virtual Environment
Venv
Evidence
Crew(stream=True)/LLM(stream=True)with a LiteLLM-routed model: the first tool call returns to the executor withid=None, so the follow-up request containstool_calls[].id = null/tool_call_id = nulland fails provider-side validation (or silently breaks call/result correlation).await llm.acall(..., tools=..., available_functions=...)with streaming enabled: the tool is never executed and the call returns"".
Pre-fix, the regression tests fail with id=None (sync and async) and with the tool function never called (async with available_functions).
Possible Solution
- Add
id: str | None = NonetoAccumulatedToolArgsand capturetool_call.idat both accumulation sites (_handle_streaming_tool_callsand the async inline loop). - Set
id=tool_arg.idwhen rebuildingChatCompletionDeltaToolCallin both_handle_streaming_responseand_ahandle_streaming_response. - In
_ahandle_streaming_response, replay the rebuilt calls against a freshdefaultdict(AccumulatedToolArgs)instead of the populated accumulator, so the arguments stay parseable and the tool executes.
Additional context
Found and written with an AI coding agent (Devin, reviewed by Claude Code); please apply the llm-generated label. A fix with offline regression tests is ready and will be linked here.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with lib/crewai/src/crewai/llm.py, especially AccumulatedToolArgs and the sync and async streaming handlers around the cited lines. Run lib/crewai/tests/llms/litellm/test_litellm_streaming_tool_calls.py first to observe the mocked streaming failures. Done means rebuilt calls preserve provider ids and the async available_functions path executes the tool with parseable arguments.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- api, backend, testing-qa
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 78/100