openai / openai/codex

app-server: active turn hangs indefinitely when the client disconnects during an in-flight exec (remote unix-socket transport)

Open
#40,306 2 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

app-server bug CLI connectivity remote tool-calls
Dominant language
Rust
Stars
125k
Forks
19.4k
PR merge metrics
PR metrics pending

Description

app-server: active turn hangs indefinitely when the client disconnects during an in-flight exec (remote unix-socket transport)

Environment

  • codex-cli / app-server 0.149.0 (standalone install), Debian 13 x86_64
  • daemon started via codex remote-control start; client is codex --remote unix://… on macOS, reaching the daemon's control socket through an SSH-forwarded unix socket
  • long-lived thread (resumed legacy session), agent mid-turn running a long shell command via the exec tool

What happened

The client machine (laptop) went to sleep, severing the SSH tunnel and therefore the websocket-over-unix-socket connection, while the agent's turn had an exec in flight (a long go test run it was monitoring).

Journal timeline (thread rollout):

  • 01:09:2101:11:03 — turn active and healthy: agent emits progress messages every ~60–75 s while polling its long-running command.
  • 01:11:03 — last journal record of the turn. Client connection died at approximately this time.
  • 01:1101:2918 minutes of total silence: no further items, no task_complete, no error, no abort. The turn produced nothing and never finished.
  • 01:29:43 — client reconnects, resumes the thread, submits a new user message (task_started + user_message).
  • 01:29:57turn_aborted recorded — i.e. the stalled turn is only terminated as a side effect of the client-side interrupt at reattach, ~19 minutes after it stopped making progress.

Expected

Per the disconnect design (thread connection_closed only unsubscribes; turns are not tied to connections), the turn should have either:

  1. continued to completion server-side (if its exec is turn-owned), or
  2. — if the in-flight exec is connection-scoped and gets reaped on disconnect — had its pending tool call resolve with an error so the model could react (retry, report, finish the turn).

Actual

The in-flight exec appears to be reaped by the connection-close cleanup, and the turn's pending await on it never resolves. The turn neither completes nor fails: it hangs silently and invisibly (an unattended daemon would hold it forever; ours held it until manual reconnect). No error is journaled at the stall point.

Source reading (0.149.0)

  • app-server/src/lib.rs TransportEvent::ConnectionClosedprocessor.connection_closed(...)
  • app-server/src/message_processor.rs connection_closed drains RPCs, then calls outgoing/fs/command_exec/process_exec/thread processors' cleanup.
  • app-server/src/request_processors/thread_processor.rs connection_closed only removes thread subscriptions (no turn abort) — consistent with the intended "turns survive disconnects" semantics.
  • command_exec.rs / process-exec cleanup terminate sessions owned by the closed connection. When the terminated execution is the one an active turn is awaiting, nothing appears to deliver a completion/error back to the turn — matching the observed permanent stall.

Suggested fix direction

When connection-close cleanup terminates an execution that an active turn is awaiting, resolve the pending tool call with an explicit error (e.g. "execution terminated: owning connection closed") so the turn can proceed and journal the outcome — or, alternatively, don't scope turn-initiated execs to the client connection at all, since the turn itself survives the disconnect by design.

Notes

  • Recovery works as expected: reconnect + interrupt + new turn continues from journaled state; no data loss beyond the stalled turn's unfinished tail.
  • Possibly adjacent to #36185 / #29262 (remote-control disconnect symptoms), but those concern notification delivery; this is about turn liveness.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start at app-server/src/lib.rs, then trace TransportEvent::ConnectionClosed through app-server/src/message_processor.rs and the cleanup handlers in command_exec.rs and process-exec. Compare that flow with thread_processor.rs connection handling and reproduce the remote disconnect during an in-flight exec. Done means the pending tool call resolves with an error, the turn can finish or fail, and the outcome is journaled instead of hanging.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
backend
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
68/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.