[Bug]: ACP session/load failure silently starts a fresh session without restoring conversation context

Problem

An existing Agent Canvas conversation can silently lose its model-side context when ACP session recovery fails. The UI retains the old messages, but the backend switches to a fresh ACP session and sends only the latest user message. The user is not clearly informed that the underlying session has changed.

This report concerns the unsafe fallback behavior, separately from the subprocess cleanup problem tracked in #4910 and PR #4909.

Environment

• Self-hosted Agent Canvas 1.17.0, Linux x86_64, Docker
• ACP provider: @agentclientprotocol/codex-acp 1.12.0 (runtime npx fallback)
• Startup log reports a pinned-provider version of 1.10.0 versus actual 1.12.0; whether this mismatch contributes is unknown.

Observed sequence

1. A long conversation was used to discuss and validate a design.
2. A follow-up message arrived while the previous async turn was still in flight.
3. OpenHands cancelled the in-flight task and restarted ACP after cancellation/drain handling.
4. Loading the original ACP session returned an Internal error.
5. OpenHands created a fresh ACP session under the same Canvas conversation and processed the follow-up without the earlier context.
6. The assistant explicitly stated that it could not see the preceding discussion, although the UI and persisted events retained it.

Sanitized logs, in order:

interrupt(): cancelled in-flight arun() task
arun() interrupted via task cancellation
Restarting ACP session after cancelled prompt drain timeout
ACP load_session(<original-session>) failed (Internal error); starting a fresh session
Sending ACP prompt (idle_timeout=1800s, blocks=2, async)

The persisted acp_session_id changed at this point.

Additional evidence and recovery

• The old Codex app-server was still alive under an orphaned ACP process tree.
• /proc/locks confirmed that this old process held a WRITE/FLOCK lock on the original thread's writer-lock file.
• The original Codex rollout file and Canvas event history still existed.
• After stopping the service to clear the stale processes and restoring the original session binding, the log reported Resumed ACP session.
• A verification turn, explicitly disallowing tool use and file reads, correctly recalled several specific facts from the earlier discussion. The event log showed no ACP tool calls in that turn.

The orphan/lock is strong evidence for the resume failure, but the original Internal error did not expose its nested cause, so this is not claimed as a fully isolated reproduction of the lock failure. The context-loss fallback itself is directly observed.

Relevant code

In openhands-sdk/openhands/sdk/agent/acp_agent.py:

• _start_acp_server catches ACPRequestError from load_session and falls through to new_session, including for an Internal error.
• _astep builds the next prompt from the supplied user message or the most recent user MessageEvent, rather than replaying the full conversation into the new session.

Inspected upstream revision: 22c85eb0e0db8f4386380d095e9fe6933af2e65f.
https://github.com/OpenHands/software-agent-sdk/blob/22c85eb0e0db8f4386380d095e9fe6933af2e65f/openhands-sdk/openhands/sdk/agent/acp_agent.py

Expected behavior

• A failed resume must not silently turn an existing conversation into a context-free session.
• Preserve the original session ID and expose a recoverable error, with a clear retry/recovery path.
• Distinguish unknown-session errors from internal, lock/ownership, and transient errors.
• If starting fresh is offered, make the loss of native context explicit and require an intentional choice; any transcript-based recovery should be clearly identified.
• Ensure interruption/restart cleans up the spawned process tree (related #4910 / #4909).

Suggested regression coverage

Simulate load_session returning an Internal error after an interrupted turn. Verify that the original session ID is retained, the conversation enters an explicit recoverable error state, and no fresh-session prompt is sent silently.

Start in openhands-sdk/openhands/sdk/agent/acp_agent.py, reading _start_acp_server and _astep alongside the suggested regression coverage. Reproduce load_session returning an Internal error after an interrupted turn, then verify that the original session ID remains, recovery is explicit, and no fresh-session prompt is sent silently.