[BUG] Native OpenAI tool-calling flow raises "Invalid response from LLM call" on a benign empty-text turn-end (tool-only-reply agents)
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 58.8k
- Forks
- 8.5k
- Avg merge
- 1d 15h
- Merged PRs (30d)
- 109
Description
Description
Using the native OpenAI provider (LLM.api == "completions", the default) with an agent that communicates only through tool calls (never a plain-text final answer — e.g. a platform-integration agent whose system prompt instructs it to reply exclusively via a custom send_message-style tool), the model reliably returns a completion with finish_reason='stop', completion_tokens=3, reasoning_tokens=0, content='', tool_calls=None immediately after a tool result that reads as "done" (e.g. a status-update tool returning completed, or the reply tool returning success).
crewai/llms/providers/openai/completion.py::_handle_completion falls through this case to content = message.content or "" (empty string), which propagates to crewai/utilities/agent_utils.py's _validate_and_finalize_llm_response, raising ValueError("Invalid response from LLM call - None or empty.") — indistinguishable from a genuine provider failure, even though this is the model correctly recognizing it has nothing further to say.
This is reproducible on effectively every turn for a tool-only-reply agent design (measured ~61/61 turns in our own integration), not an occasional flake.
Steps to Reproduce
from crewai import LLM
llm = LLM(model="gpt-5.4-mini") # native OpenAI provider, api="completions" (default)
tools = [{
"type": "function",
"function": {
"name": "reply",
"description": "Send the only allowed response to the user.",
"parameters": {
"type": "object",
"properties": {"text": {"type": "string"}},
"required": ["text"],
},
},
}]
messages = [
{"role": "system", "content": "You may only respond by calling the `reply` tool. Never produce plain text."},
{"role": "user", "content": "Say hello."},
{"role": "assistant", "content": None, "tool_calls": [{"id": "call_1", "type": "function", "function": {"name": "reply", "arguments": '{"text": "hello"}'}}]},
{"role": "tool", "tool_call_id": "call_1", "name": "reply", "content": '{"status": "success"}'},
]
# The next call -- asking "anything else?" after a terminal-looking tool result --
# reliably comes back empty (finish_reason='stop', ~3 completion tokens, no
# content, no tool_calls), which _handle_completion turns into the ValueError.
llm.call(messages, tools=tools)
Expected behavior
A turn that ends with no text and no further tool calls, immediately after a tool result the model reasonably reads as terminal, should be distinguishable from a genuine empty/failed provider response -- e.g. via finish_reason ('stop' with reasoning_tokens=0, vs. 'length' or an actual API error) -- rather than always raising the same generic ValueError. Agent designs that reply exclusively through tool calls (increasingly common for platform/tool-integrated agents) have no legitimate way to produce non-empty final text, so this forced-final-answer step can never succeed for them without an external workaround.
Environment
- crewAI 1.15.5, native OpenAI provider (not litellm)
- Model:
gpt-5.4-mini - OS: reproduced on both Linux and Windows
Additional context
We work around this in our own SDK by catching the ValueError and treating it as a finished turn once some tool has already run that turn -- but the case where this happens on a turn's first LLM call (before any tool has run) can't safely be told apart from a real failure from outside CrewAI, so it still fails the turn. Only CrewAI has the visibility (raw finish_reason/usage) to distinguish "nothing left to say" from "the provider actually broke."
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in crewai/llms/providers/openai/completion.py::_handle_completion and trace how the response reaches crewai/utilities/agent_utils.py::_validate_and_finalize_llm_response. Reproduce the tool-only conversation with the provided Python example, then verify that a benign stopped empty response is distinguished from a genuine empty or failed provider response without changing normal tool-call handling.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- ai
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 72/100