OpenHands / OpenHands/software-agent-sdk

Activating an LLM profile via /api/profiles/{name}/activate doesn't update AgentProfile.llm_profile_ref, allowing the two to drift silently

Open
#4,338 2 comments 0 reactions 1 assignee View on GitHub

@VascoSch92 is already working on this.

Since Aug 3, 2026.

architecture bug llm priority:medium Stale
Dominant language
Python
Stars
1.1k
Forks
539
Avg merge
1d 19h
Merged PRs (30d)
137

Description

Bug Description

POST /api/profiles/{name}/activate (the LLM-profile activation endpoint) updates
agent_settings.llm and active_profile atomically. POST /api/agent-profiles/{id}/activate (the AgentProfile activation endpoint) is, by
contract, pointer-only and must never touch agent_settings
(ActivateAgentProfileResponse.agent_settings_applied is hardcoded False for
exactly this reason). Nothing reconciles the two: activating an LLM profile never
updates any AgentProfile.llm_profile_ref that pointed at the previously-active
profile, so a seeded/well-known default AgentProfile can end up permanently
referencing a stale or deleted LLM profile while the system's actual active LLM
profile has moved on.

Root Cause

# openhands-agent-server/openhands/agent_server/profiles_router.py
def apply_profile(settings: PersistedSettings) -> PersistedSettings:
    settings.agent_settings = settings.agent_settings.model_copy(update={"llm": llm})
    settings.active_profile = name
    return settings

versus

# openhands-agent-server/openhands/agent_server/agent_profiles_router.py
"""
...Activation here is pointer-only — unlike the LLM ``/activate`` it must
**not** write ``agent_settings`` (the creation-time-only contract).
"""
...
class ActivateAgentProfileResponse(BaseModel):
    ...
    agent_settings_applied: bool = False  # Always False: activation is pointer-only by contract.

resolve_agent_profile() (the only code path that actually re-resolves
llm_profile_ref into a real LLM config) is invoked from
conversation_service.py's _resolve_agent_from_profile, which only runs when a
conversation is launched by profile id. The much more common path — a client
fetching GET /api/settings and forwarding agent_settings to POST /api/conversations (the pattern used by this repo's own agent-server automations,
and by the local/well-known default AgentProfile launch path per #16193 in
OpenHands/OpenHands) — never touches resolve_agent_profile and therefore never
notices that llm_profile_ref is stale.

Steps to Reproduce

  1. Save two LLM profiles, e.g. POST /api/profiles/profile-a and POST /api/profiles/profile-b.
  2. POST /api/profiles/profile-a/activateactive_profile is now profile-a.
  3. Confirm (or create) an AgentProfile named default whose llm_profile_ref
    is profile-a (this is the seeded default in a fresh instance).
  4. POST /api/profiles/profile-b/activateactive_profile is now profile-b,
    agent_settings.llm now reflects profile-b.
  5. GET /api/agent-profiles/defaultllm_profile_ref still reads profile-a.
  6. DELETE /api/profiles/profile-a — succeeds (no FK violation, since nothing
    currently guards default's specific well-known status the way named profiles
    are guarded elsewhere).
  7. GET /api/agent-profiles/defaultllm_profile_ref now names a profile
    that no longer exists
    , with no error, warning, or repair anywhere in the
    response.

Actual Impact Observed

Reproduced this drift on a live deployment: the active LLM profile and the
default AgentProfile's llm_profile_ref had silently diverged (two different
named profiles, two different underlying models), with no error surfaced anywhere
in GET /api/settings or GET /api/agent-profiles. This is the same root cause
independently reported (and only partially fixed — frontend gate logic only, per
the shipped PR) in OpenHands/OpenHands#16193, whose own "Suggested fix" section
explicitly flagged this as an unaddressed follow-up:

Keeping the seeded default.llm_profile_ref synchronized when the active LLM
profile changes may also reduce stale state, but the frontend gate should still
agree with the launch semantics.

See also #3841 (LLM/AgentProfile identity asymmetry — related but distinct: that
issue is about rename-safety, this one is about activation-time sync) and #4314
(a different, now-fixed desync path between PATCH /api/settings and
agent_settings.llm — same category of bug, different trigger).

Possible Fix Directions

Not prescribing one — this is a real design decision:

  • (a) When an LLM profile is deactivated (a different one is activated) or
    deleted, cascade-update any AgentProfile.llm_profile_ref that pointed at it —
    at minimum for the seeded default profile, which is already special-cased
    elsewhere (see OpenHands/OpenHands#16193/#16200) as "the enriched baseline that
    launches through live agent_settings."
  • (b) Expose a computed staleness flag on GET /api/agent-profiles/{id} (e.g.
    llm_profile_ref_matches_active: bool) so callers can detect drift without
    reimplementing the diff themselves.
  • (c) Auto-/materialize (or an equivalent dry-run resolve) as part of the
    GET /api/agent-profiles/{id} response so staleness is visible without a
    separate call.
  • (d) Guard DELETE /api/profiles/{name} for default's current
    llm_profile_ref the same way it already guards other referenced profiles
    (the FK check in delete_llm_profile exists — worth checking why it didn't
    catch this).

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.