OpenHands / OpenHands/software-agent-sdk

[Bug]: ACP mode: conversation title generation fails with missing credentials (explicit LLM without a key)

Open
#4,867 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

acp bug llm priority:medium ready-for-dev
Dominant language
Python
Stars
1.1k
Forks
539
Avg merge
1d 19h
Merged PRs (30d)
137

Description

Is there an existing issue for the same bug?
  • I have searched existing issues and this is not a duplicate.
Bug Description

In ACP mode the conversation is driven by an external subprocess and the LLM settings are inert, but conversation title generation still makes a real LLM call.

ACPAgent installs a sentinel LLM (openhands-sdk/openhands/sdk/agent/acp_agent.py:264-269):

ACP_SENTINEL_USAGE_ID = "acp-managed"

def _make_dummy_llm() -> LLM:
    """Create a dummy LLM that should never be called directly."""
    return LLM(model="acp-managed", usage_id=ACP_SENTINEL_USAGE_ID)

That is only a default. A backend that persists an agent LLM — an agent profile with a model set — keeps that model on the agent while the conversation runs in the ACP subprocess under its own credentials. The sentinel guard in generate_conversation_title matches on usage_id == "acp-managed", so with a persisted LLM it never fires and titling reaches a real provider with no key.

The auto-title fallback has no ACP check (openhands-agent-server/openhands/agent_server/conversation_service.py):

title_llm = self._load_title_llm()
if title_llm is None:
    title_llm = conversation.agent.llm if conversation else None

Agent Canvas documents the same contract from the other side (src/types/settings.ts:105): for agent_kind: "acp", "The LLM / condenser settings are inert; mcp_config and acp_command / acp_args / acp_model / acp_server apply instead."

Expected Behavior

With an ACP agent and no explicitly configured title_llm_profile, skip LLM titling and fall back to truncation. No provider call, no retries, no error surfaced to the UI.

Actual Behavior

The agent reports agent_kind == "acp" while carrying a real model whose usage_id is not the sentinel, so the guard that is supposed to prevent the call cannot fire. Save this as acp_repro.py in a checkout of this repository and run it:

make build
uv run python acp_repro.py

Output on main:

agent_kind            : acp
agent.llm.model       : gpt-4o
agent.llm.usage_id    : agent
sentinel usage_id     : acp-managed
sentinel guard fires  : False

Titling therefore calls a provider with no credentials, retries four times, and surfaces the failure.

Steps to Reproduce
  1. Run an agent-server with agent_kind: "acp" and acp_server: "claude-code" under subscription auth.
  2. Leave title_llm_profile unset, and set an active_agent_profile_id whose profile carries a model.
  3. Start a conversation and send any first user message.
  4. Auto-titling calls that model with no key: four retries, and since #16686 the error is published to the frontend.
Acceptance Criteria
  • With agent_kind == "acp" and no title_llm_profile, auto-titling makes no LLM call.
  • The conversation title falls back to truncation in that case.
  • An explicitly configured title_llm_profile still takes precedence and is used.
  • Non-ACP agents keep the existing agent.llm fallback unchanged.
  • No credentials error is surfaced to the frontend for an ACP backend that holds no model credentials.
Installation Method

Self-hosted openhands-agent-server released image; reproduced from a source checkout via make build and uv run.

If you selected "Other", please specify

No response

SDK Version

1.44.0 (observed). Reproduced against main.

Version Confirmation
  • I have confirmed this bug exists on the LATEST version of OpenHands SDK
Python Version

3.13.15

Model Name (if applicable)

Conversation runs under claude-code via ACP subscription auth; the agent profile carried an OpenAI model, which is the one titling tried to call.

Operating System

Linux

Logs and Error Messages

Self-hosted worker, agent-server 1.44.0, agent_kind: "acp", acp_server: "claude-code", subscription auth. The conversation itself works and MCP works; every conversation logs four retries:

[agent-server] ERROR   litellm.InternalServerError: InternalServerError: OpenAIException -
  Missing credentials. Please pass an `api_key`, `workload_identity`, `admin_api_key`,
  or set the `OPENAI_API_KEY` or `OPENAI_ADMIN_KEY` environment variable.. Attempt #3
[agent-server] WARNING Error generating conversation title with LLM: ...  title_utils.py:152

The failure is already non-fatal — generate_conversation_title catches, logs, and callers fall back to truncation — but since #16686 the error is also propagated to the frontend, so ACP users see a credentials error on a backend that deliberately holds no model credentials. Cost is roughly 15s of retries per conversation plus a misleading error.

Minimal Code Sample
"""ACP agent LLM is inert, yet auto-titling still resolves to it."""
from openhands.sdk.agent import ACPAgent
from openhands.sdk.agent.acp_agent import ACP_SENTINEL_USAGE_ID
from openhands.sdk.llm import LLM

# A backend that persists an agent LLM (an agent profile with a model set)
# keeps that model on the agent even though ACP runs the conversation.
agent = ACPAgent(
    llm=LLM(model="gpt-4o", usage_id="agent"),
    acp_server="claude-code",
    acp_command=["claude-agent-acp"],
)
print("agent_kind            :", agent.agent_kind)
print("agent.llm.model       :", agent.llm.model)
print("agent.llm.usage_id    :", agent.llm.usage_id)
print("sentinel usage_id     :", ACP_SENTINEL_USAGE_ID)
print("sentinel guard fires  :", agent.llm.usage_id == ACP_SENTINEL_USAGE_ID)
Screenshots and Additional Context

Note for anyone hitting this: the obvious workaround — exporting OPENAI_API_KEY for the agent-server — is a trap. The ACP subprocesses inherit the parent environment, and the Codex CLI prefers OPENAI_API_KEY when present, which can silently switch the main workload from subscription auth to API auth.

A PR is open at #4868.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with openhands-agent-server/openhands/agent_server/conversation_service.py and the title-generation path referenced at title_utils.py:152; review ACPAgent in openhands-sdk/openhands/sdk/agent/acp_agent.py and the ACP contract in src/types/settings.ts. Run the supplied acp_repro.py after make build and uv run, then verify the acceptance criteria for explicit title profiles, ACP fallback, and non-ACP agents.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
api, backend
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.