OpenHands / OpenHands/software-agent-sdk

[Bug]: RouterLLM bypasses model selection for async completions

Open
#4,660 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

architecture bug llm priority:medium ready-for-dev
Dominant language
Python
Stars
1.1k
Forks
539
Avg merge
1d 19h
Merged PRs (30d)
137

Description

Bug Description

RouterLLM overrides only synchronous completion() in openhands-sdk/openhands/sdk/llm/router/base.py:57-95. Normal async agent execution calls inherited LLM.acompletion(), so routing selection is never performed and the router's placeholder model is used instead of a configured child.

The synchronous path also writes the selected child into shared Pydantic field active_llm before dispatch. Concurrent calls can overwrite one another between selection and use. Meanwhile __getattr__() at :113-117 ignores active_llm and always delegates to the first configured model.

Expected Behavior

Sync and async calls should perform identical per-request routing, without dispatch depending on mutable shared selection state.

Actual Behavior

An independent, self-contained reproducer is posted in the issue comments. Save
it as /tmp/repro_4660.py, then run:

uv run python /tmp/repro_4660.py

On SDK commit 6d3881035, it exercises the real inherited async completion path and real TestLLM children; an intentionally unreachable local endpoint makes wrong dispatch deterministic and prints:

async_error_type=LLMBadRequestError
attempted_model=openai/router-placeholder
select_llm_calls=0
selected_child_remaining=1
Suggested Implementation

Resolve the selected child into a call-local variable through one shared helper, then delegate immediately from both completion() and acompletion(). Keep active_llm only as a compatibility field/accessor for now; removing it is public API work and requires the SDK deprecation runway.

Acceptance Criteria
  • Async completions select and invoke the expected child LLM.
  • Sync and async paths share selection/validation behavior.
  • Concurrent calls selecting different children cannot cross-dispatch.
  • Token callbacks, call context, tools, retries, and kwargs reach the selected child unchanged.
  • Invalid selector keys fail deterministically.
  • Existing public constructor/serialization compatibility is preserved.
Version

Current main at 6d3881035. Existing searches found no open issue covering async router bypass.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in openhands-sdk/openhands/sdk/llm/router/base.py, especially completion(), the inherited acompletion() path, and getattr; run /tmp/repro_4660.py with uv run python /tmp/repro_4660.py. Done means sync and async calls share selection behavior, concurrent requests dispatch to their own selected children, and callbacks, context, tools, retries, kwargs, invalid selectors, and public compatibility remain correct.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend-api-design
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
68/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.