crewAIInc / crewAIInc/crewAI

[BUG] Native OpenAI tool-calling flow raises "Invalid response from LLM call" on a benign empty-text turn-end (tool-only-reply agents)

Open
#7,349 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
58.8k
Forks
8.5k
Avg merge
1d 15h
Merged PRs (30d)
109

Description

Description

Using the native OpenAI provider (LLM.api == "completions", the default) with an agent that communicates only through tool calls (never a plain-text final answer — e.g. a platform-integration agent whose system prompt instructs it to reply exclusively via a custom send_message-style tool), the model reliably returns a completion with finish_reason='stop', completion_tokens=3, reasoning_tokens=0, content='', tool_calls=None immediately after a tool result that reads as "done" (e.g. a status-update tool returning completed, or the reply tool returning success).

crewai/llms/providers/openai/completion.py::_handle_completion falls through this case to content = message.content or "" (empty string), which propagates to crewai/utilities/agent_utils.py's _validate_and_finalize_llm_response, raising ValueError("Invalid response from LLM call - None or empty.") — indistinguishable from a genuine provider failure, even though this is the model correctly recognizing it has nothing further to say.

This is reproducible on effectively every turn for a tool-only-reply agent design (measured ~61/61 turns in our own integration), not an occasional flake.

Steps to Reproduce
from crewai import LLM

llm = LLM(model="gpt-5.4-mini")  # native OpenAI provider, api="completions" (default)

tools = [{
    "type": "function",
    "function": {
        "name": "reply",
        "description": "Send the only allowed response to the user.",
        "parameters": {
            "type": "object",
            "properties": {"text": {"type": "string"}},
            "required": ["text"],
        },
    },
}]

messages = [
    {"role": "system", "content": "You may only respond by calling the `reply` tool. Never produce plain text."},
    {"role": "user", "content": "Say hello."},
    {"role": "assistant", "content": None, "tool_calls": [{"id": "call_1", "type": "function", "function": {"name": "reply", "arguments": '{"text": "hello"}'}}]},
    {"role": "tool", "tool_call_id": "call_1", "name": "reply", "content": '{"status": "success"}'},
]

# The next call -- asking "anything else?" after a terminal-looking tool result --
# reliably comes back empty (finish_reason='stop', ~3 completion tokens, no
# content, no tool_calls), which _handle_completion turns into the ValueError.
llm.call(messages, tools=tools)
Expected behavior

A turn that ends with no text and no further tool calls, immediately after a tool result the model reasonably reads as terminal, should be distinguishable from a genuine empty/failed provider response -- e.g. via finish_reason ('stop' with reasoning_tokens=0, vs. 'length' or an actual API error) -- rather than always raising the same generic ValueError. Agent designs that reply exclusively through tool calls (increasingly common for platform/tool-integrated agents) have no legitimate way to produce non-empty final text, so this forced-final-answer step can never succeed for them without an external workaround.

Environment
  • crewAI 1.15.5, native OpenAI provider (not litellm)
  • Model: gpt-5.4-mini
  • OS: reproduced on both Linux and Windows
Additional context

We work around this in our own SDK by catching the ValueError and treating it as a finished turn once some tool has already run that turn -- but the case where this happens on a turn's first LLM call (before any tool has run) can't safely be told apart from a real failure from outside CrewAI, so it still fails the turn. Only CrewAI has the visibility (raw finish_reason/usage) to distinguish "nothing left to say" from "the provider actually broke."

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in crewai/llms/providers/openai/completion.py::_handle_completion and trace how the response reaches crewai/utilities/agent_utils.py::_validate_and_finalize_llm_response. Reproduce the tool-only conversation with the provided Python example, then verify that a benign stopped empty response is distinguished from a genuine empty or failed provider response without changing normal tool-call handling.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
ai
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
72/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.