openai / openai/codex

Linux: a locked Secret Service keyring makes new sessions unusable and hangs exit forever

Open
#41,031 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

auth bug CLI mcp tool-calls
Dominant language
Rust
Stars
125k
Forks
19.4k
PR merge metrics
PR metrics pending

Description

What version of Codex CLI is running? 0.149.1

What subscription do you have? ChatGPT Pro

What platform is your computer? Linux 7.1.9 x86_64 (Fedora 44, GNOME/Wayland, gnome-keyring 50.0)

What terminal emulator? kitty (no multiplexer)

Codex doctor report not available

What issue are you seeing?

On Linux, MCP OAuth credentials go to the Secret Service keyring by default
(mcp_oauth_credentials_store defaults to Auto). If the keyring collection is
locked, every credential read blocks forever, and Codex has no defense at any
layer. This produces two distinct failures from one cause:

A. Exit hangs forever. Quitting never returns to the shell prompt. The process
does not exit and does not respond. Only kill clears it.

B. New sessions cannot accept input at all. A freshly started codex never
becomes usable — typing anything just shows Queued follow-up inputs and no LLM
call is ever made. Restarting does not help: every new process hits the same wall.

Once in this state the CLI is effectively unusable until the keyring is unlocked
externally, and "just restart it" — the natural user reaction — makes things worse,
because each restart leaves behind another stuck blocking thread.

Root cause

The blocking call chain:

create_pending_transport()                 rmcp-client/src/rmcp_client.rs:987 (async fn)
  └─ tokio::task::spawn_blocking(...)      rmcp-client/src/rmcp_client.rs:1040
      └─ DefaultKeyringStore::load(...)    keyring-store/src/lib.rs:43
          └─ keyring "linux-native-async-persistent"
              └─ keyutils cache miss → falls back to secret-service
                  └─ secret_service::blocking::SecretService::connect()
                      └─ zbus::blocking::Connection::session()
                          └─ blocks forever on the locked collection

Three independent gaps turn that into the two failures above:

1. Runtime::drop waits for blocking tasks with no timeout → failure A.

build_runtime() (arg0/src/lib.rs:290) builds a runtime that is later dropped
normally. Tokio's Runtime::drop calls the blocking pool's shutdown(None), which
does shutdown_rx.wait(None) and then joins every worker — an unbounded wait. Any
blocking task that never returns pins the process forever.

2. tokio::time::timeout cannot cancel spawn_blocking → the timeout is a no-op.

codex-mcp/src/rmcp_client.rs:330 wraps make_rmcp_client in a 30s
DEFAULT_STARTUP_TIMEOUT (:92). But that only bounds the await side. The closure
handed to spawn_blocking has no cancellation point, so the thread stays stuck on
D-Bus regardless. The timeout does not fix the hang — it hides it: startup reports
an error and moves on while a stuck thread is silently left in the pool. With
transport retries (STREAMABLE_HTTP_RETRY_DELAYS_MS,
rmcp-client/src/streamable_http_retry.rs:23) and reconnects, these accumulate,
and every one of them is later joined by gap 1.

3. Unreachable credentials block session readiness → failure B.

Input is queued because the session is not configured yet:

// tui/src/chatwidget/input_submission.rs:105
if !self.is_session_configured() {
    tracing::warn!("cannot submit user message before session is configured; queueing");

Session readiness waits on MCP startup, and MCP startup waits on the keyring. The
codebase already recognizes this class of risk for a different input source:

// core/src/thread_manager.rs:1901
// Enable Full Access form input only after session startup so a required MCP server cannot
// block startup while waiting for form input.

The same protection was never extended to credential storage. An MCP server whose
OAuth token cannot be read should degrade to "this server is unavailable", not take
the whole conversation down with it.

What steps can reproduce the bug?

  1. Linux desktop with a Secret Service provider (gnome-keyring / KWallet).
  2. Configure one or more remote HTTP MCP servers that authenticate via OAuth, and
    sign in so the tokens land in the keyring. Leave mcp_oauth_credentials_store
    at its default (Auto), and make sure $CODEX_HOME/.credentials.json does not
    exist — i.e. the credentials really are keyring-only.
  3. Lock the keyring collection:
    busctl --user call org.freedesktop.secrets /org/freedesktop/secrets \
      org.freedesktop.Secret.Service Lock ao 1 \
      /org/freedesktop/secrets/collection/login
    
    Verify with:
    busctl --user get-property org.freedesktop.secrets \
      /org/freedesktop/secrets/collection/login \
      org.freedesktop.Secret.Collection Locked
    # b true
    
  4. Start codex and type anything → failure B: the input is queued as
    Queued follow-up inputs, no LLM call happens.
  5. Quit that session → failure A: the process never exits.

Note: in the incident that surfaced this, step 3 happened by itself —
gnome-keyring-daemon crashed (a known upstream gnome-keyring NULL-deref during
Secret Service session negotiation) and the D-Bus-activated replacement came up
without the login password, so the collection was locked. The explicit Lock call
above is offered as a deterministic way to reach the same state; the observations
below
come from the real incident, not from the synthetic Lock.

Observed evidence (from the real incident)

A long-running session that touched MCP OAuth three times while the keyring was
locked, then was asked to quit:

  • Thread count dropped from 41 to 9 — shutdown got most of the way, then stopped.
  • The codex-code-mode-host child had already exited; history had already been
    flushed to disk. Only the runtime teardown was left.
  • The 9 survivors included three tokio-rt-worker threads, each paired with a
    zbus::Connection thread
    created 0–5s apart, matching the three keyring touches.
  • Those blocking threads had been alive for 892s / 985s / 1553s, against tokio's
    10s blocking-pool KEEP_ALIVE — i.e. not idle, still executing.
  • CPU time was frozen (blocked, not spinning). Three D-Bus connections to the
    session bus were ESTAB with empty send/receive queues.
  • Independent confirmation of where it blocks: the same Secret Service read via
    secret-tool never returned within 150s while locked, and returned 10 records
    in 9ms once unlocked
    .
  • Unlocking the keyring did not revive the hung process — Runtime::drop was
    already inside the join. It had to be killed. (SIGTERM is enough; the process
    does not catch it.)

Two other sessions started after the keyring locked showed failure B. They had no
obviously stuck threads, which makes B easy to misdiagnose as healthy — but each
dropped 6 threads the instant the keyring was unlocked.

What is the expected behavior?

A locked or unavailable credential store should degrade, not deadlock:

  1. Exit should always terminate. Use shutdown_timeout(...) instead of a bare
    Runtime::drop, so a stuck blocking task cannot pin the process. This alone
    fixes failure A.
  2. Keyring calls should have their own deadline. DefaultKeyringStore::load/save
    passes straight through to a synchronous API with no time bound. Since an outer
    tokio::time::timeout provably cannot cancel spawn_blocking, the bound has to
    live on the blocking call itself. This is the root fix.
  3. Credential failure should not block session readiness. An MCP server with an
    unreadable token should be marked unavailable and the session should still become
    usable, the same way thread_manager.rs:1901 already protects startup from a
    server waiting on form input. This fixes failure B.

Ideally the user would also see why — something like "MCP server X: credential
store is locked" — instead of a silent queue with no explanation.

Additional information

  • Line numbers may shift between patch releases; the quoted snippets and function
    names should locate the code regardless.

  • Dependency versions in play: keyring 3.6.3 (feature
    linux-native-async-persistent), secret-service 4.0.0, zbus 4.4.0,
    tokio 1.53.

  • Both upstreams explicitly warn against exactly this usage:

    • secret-service 4.0.0, src/blocking/mod.rs: "It is important to not call
      this these functions in an async context or otherwise the runtime may stall."
    • zbus 4.4.0, src/blocking/mod.rs: "you must not use them in async contexts
      because of the infamous async sandwich footgun"

    keyring's async-secret-service feature nevertheless routes through
    secret_service::blocking (keyring-3.6.3/src/secret_service.rs:851), so Codex
    inherits a blocking D-Bus client under an async runtime.

  • Workaround for anyone hitting this: unlock the keyring (on GNOME, locking and
    unlocking the screen re-supplies the password to the running daemon), then kill
    any already-hung process. Setting mcp_oauth_credentials_store = "file" avoids
    the keyring entirely, at the cost of storing OAuth tokens in a plaintext file.

  • Possibly related in spirit: the same synchronous-keyring-inside-async pattern
    appears in refresh_and_persist_chatgpt_token
    (login/src/auth/manager.rs), which calls persist_tokens directly from an
    async fn, whereas the login path deliberately wraps the same logic in
    spawn_blocking ("Reuse existing synchronous logic but run it off the async
    runtime", login/src/server.rs). That path was not involved here because CLI auth
    defaults to file storage, but it would block an async worker rather than a
    blocking-pool thread if it ever hit a locked keyring.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with build_runtime in arg0/src/lib.rs, the credential load path in keyring-store/src/lib.rs, and MCP startup in rmcp-client/src/rmcp_client.rs and codex-mcp/src/rmcp_client.rs. Read the session readiness check in tui/src/chatwidget/input_submission.rs and the startup protection in core/src/thread_manager.rs. Done means locked credentials cannot block session usability or process exit, and the failure is surfaced as an unavailable MCP server.

Written by the indexing model from the issue text.

Assessment

Tech stack
linux, rust
Domain
authentication, backend, cli
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.