openai / openai/codex

Dropped app-server events corrupt resumed Codex sessions

Open
#38,234 4 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

app-server bug CLI exec
Dominant language
Rust
Stars
125k
Forks
19.5k
PR merge metrics
PR metrics pending

Description

What version of Codex CLI is running?

codex-cli 0.147.0

What subscription do you have?

$20/mo

Which model were you using?

gpt-5.6-sol

What platform is your computer?

Darwin 25.0.0 arm64 arm

What terminal emulator and version are you using (if applicable)?

Ghostty 1.2.0; no multiplexer

Codex doctor report
{
  "schemaVersion": 1,
  "generatedAt": "1786558899s since unix epoch",
  "overallStatus": "ok",
  "codexVersion": "0.147.0",
  "checks": {
    "app_server.status": {
      "id": "app_server.status",
      "category": "app-server",
      "status": "ok",
      "summary": "background server is not running",
      "details": {
        "control socket": "/Users/REDACTED/.codex/app-server-control/app-server-control.sock",
        "daemon state dir": "/Users/REDACTED/.codex/app-server-daemon",
        "mode": "ephemeral",
        "pid file": "/Users/REDACTED/.codex/app-server-daemon/app-server.pid (missing)",
        "settings": "/Users/REDACTED/.codex/app-server-daemon/settings.json (missing)",
        "status": "not running",
        "update-loop pid file": "/Users/REDACTED/.codex/app-server-daemon/app-server-updater.pid (missing)"
      },
      "remediation": null,
      "durationMs": 6
    },
    "auth.credentials": {
      "id": "auth.credentials",
      "category": "auth",
      "status": "ok",
      "summary": "auth is configured",
      "details": {
        "auth file": "/Users/REDACTED/.codex/auth.json",
        "auth storage mode": "File",
        "stored API key": "false",
        "stored ChatGPT tokens": "true",
        "stored agent identity": "false",
        "stored auth mode": "chatgpt"
      },
      "remediation": null,
      "durationMs": 0
    },
    "config.load": {
      "id": "config.load",
      "category": "config",
      "status": "ok",
      "summary": "config loaded",
      "details": {
        "CODEX_HOME": "/Users/REDACTED/.codex",
        "config.toml": "/Users/REDACTED/.codex/config.toml",
        "config.toml parse": "ok",
        "cwd": "/Users/REDACTED/mystuff/Coding/cascade-browser",
        "enabled feature flags": "shell_tool, view_image, unified_exec, shell_snapshot, code_mode_host, terminal_resize_reflow, sqlite, hooks, enable_request_compression, multi_agent, apps, tool_search_always_defer_mcp_tools, tool_suggest, plugins, in_app_browser, in_app_updates, browser_use, browser_use_full_cdp_access, browser_use_external, computer_use, remote_plugin, plugin_sharing, image_generation, resize_all_images, item_ids, skill_mcp_dependency_install, skill_search, mentions_v2, steer, guardian_approval, goals, collaboration_modes, tool_call_mcp_elicitation, auth_elicitation, personality, fast_mode, tui_app_server, remote_compaction_v2, workspace_dependencies",
        "feature flag overrides": "none",
        "feature flags enabled": "39",
        "log dir": "/Users/REDACTED/.codex/log",
        "mcp servers": "2",
        "model": "gpt-5.6-luna",
        "model provider": "openai",
        "sqlite home": "/Users/REDACTED/.codex"
      },
      "remediation": null,
      "durationMs": 1
    },
    "git.environment": {
      "id": "git.environment",
      "category": "git",
      "status": "ok",
      "summary": "git version 2.50.1 (Apple Git-155)",
      "details": {
        ".git entry": "directory",
        "PATH git #1": "/usr/bin/git",
        "PATH git entries": "1",
        "git branch": "master",
        "git build options": "git version 2.50.1 (Apple Git-155); cpu: arm64; no commit associated with this build; sizeof-long: 8; sizeof-size_t: 8; shell-path: /bin/sh; feature: fsmonitor--daemon; libcurl: 8.7.1; zlib: 1.2.12; SHA-1: SHA1_DC; SHA-256: SHA256_BLK",
        "git exec path": "/Applications/Xcode.app/Contents/Developer/usr/libexec/git-core",
        "git version": "git version 2.50.1 (Apple Git-155)",
        "repo detected": "true",
        "repo root": "/Users/REDACTED/mystuff/Coding/cascade-browser",
        "selected git": "/usr/bin/git"
      },
      "remediation": null,
      "durationMs": 1515
    },
    "installation": {
      "id": "installation",
      "category": "install",
      "status": "ok",
      "summary": "installation looks consistent",
      "details": {
        "PATH codex #1": "/opt/homebrew/bin/codex",
        "current executable": "/opt/homebrew/lib/node_modules/@openai/codex/node_modules/@openai/codex-darwin-arm64/vendor/aarch64-apple-darwin/bin/codex",
        "install context": "npm (package /opt/homebrew/lib/node_modules/@openai/codex/node_modules/@openai/codex-darwin-arm64/vendor/aarch64-apple-darwin, bin /opt/homebrew/lib/node_modules/@openai/codex/node_modules/@openai/codex-darwin-arm64/vendor/aarch64-apple-darwin/bin, resources /opt/homebrew/lib/node_modules/@openai/codex/node_modules/@openai/codex-darwin-arm64/vendor/aarch64-apple-darwin/codex-resources, path /opt/homebrew/lib/node_modules/@openai/codex/node_modules/@openai/codex-darwin-arm64/vendor/aarch64-apple-darwin/codex-path)",
        "managed by bun": "false",
        "managed by npm": "true",
        "managed by pnpm": "false",
        "managed package root": "/opt/homebrew/lib/node_modules/@openai/codex",
        "npm update target": "/opt/homebrew/lib/node_modules/@openai/codex"
      },
      "remediation": null,
      "durationMs": 1738
    },
    "mcp.config": {
      "id": "mcp.config",
      "category": "mcp",
      "status": "ok",
      "summary": "MCP configuration is locally consistent",
      "details": {
        "configured servers": "2",
        "disabled servers": "1",
        "stdio servers": "2"
      },
      "remediation": null,
      "durationMs": 0
    },
    "network.env": {
      "id": "network.env",
      "category": "network",
      "status": "ok",
      "summary": "network-related environment looks readable",
      "details": {
        "proxy env vars": "none"
      },
      "remediation": null,
      "durationMs": 0
    },
    "network.provider_reachability": {
      "id": "network.provider_reachability",
      "category": "reachability",
      "status": "ok",
      "summary": "active provider endpoints are reachable over HTTP",
      "details": {
        "ChatGPT base URL": "https://chatgpt.com/backend-api/ reachable (HTTP 403)",
        "reachability mode": "ChatGPT auth"
      },
      "remediation": null,
      "durationMs": 1733
    },
    "network.websocket_reachability": {
      "id": "network.websocket_reachability",
      "category": "websocket",
      "status": "ok",
      "summary": "Responses WebSocket handshake succeeded",
      "details": {
        "DNS": "2 IPv4, 0 IPv6, first IPv4",
        "auth mode": "chatgpt",
        "connect timeout": "15000 ms",
        "endpoint": "wss://chatgpt.com/backend-api/<redacted>",
        "handshake result": "HTTP 101 Switching Protocols",
        "model provider": "openai",
        "models etag present": "true",
        "provider name": "OpenAI",
        "proxy env vars": "none",
        "reasoning header": "false",
        "server model present": "false",
        "supports websockets": "true",
        "wire API": "responses"
      },
      "remediation": null,
      "durationMs": 2412
    },
    "runtime.provenance": {
      "id": "runtime.provenance",
      "category": "runtime",
      "status": "ok",
      "summary": "running npm on macos-aarch64",
      "details": {
        "commit": "unknown",
        "current executable": "/opt/homebrew/lib/node_modules/@openai/codex/node_modules/@openai/codex-darwin-arm64/vendor/aarch64-apple-darwin/bin/codex",
        "install method": "npm (package /opt/homebrew/lib/node_modules/@openai/codex/node_modules/@openai/codex-darwin-arm64/vendor/aarch64-apple-darwin, bin /opt/homebrew/lib/node_modules/@openai/codex/node_modules/@openai/codex-darwin-arm64/vendor/aarch64-apple-darwin/bin, resources /opt/homebrew/lib/node_modules/@openai/codex/node_modules/@openai/codex-darwin-arm64/vendor/aarch64-apple-darwin/codex-resources, path /opt/homebrew/lib/node_modules/@openai/codex/node_modules/@openai/codex-darwin-arm64/vendor/aarch64-apple-darwin/codex-path)",
        "platform": "macos-aarch64",
        "version": "0.147.0"
      },
      "remediation": null,
      "durationMs": 0
    },
    "runtime.search": {
      "id": "runtime.search",
      "category": "search",
      "status": "ok",
      "summary": "search is OK (bundled)",
      "details": {
        "search command": "/opt/homebrew/lib/node_modules/@openai/codex/node_modules/@openai/codex-darwin-arm64/vendor/aarch64-apple-darwin/codex-path/rg",
        "search command readiness": "file exists",
        "search provider": "bundled"
      },
      "remediation": null,
      "durationMs": 17
    },
    "sandbox.helpers": {
      "id": "sandbox.helpers",
      "category": "sandbox",
      "status": "ok",
      "summary": "sandbox configuration is readable",
      "details": {
        "approval policy": "OnRequest",
        "codex-linux-sandbox helper": "none",
        "execve wrapper helper": "/Users/REDACTED/.codex/tmp/arg0/codex-arg0msOPmE/codex-execve-wrapper",
        "filesystem sandbox": "restricted",
        "network sandbox": "restricted"
      },
      "remediation": null,
      "durationMs": 0
    },
    "state.paths": {
      "id": "state.paths",
      "category": "state",
      "status": "ok",
      "summary": "state paths and databases are inspectable",
      "details": {
        "CODEX_HOME": "/Users/REDACTED/.codex (dir)",
        "active rollout files": "128 files, 211775217 total bytes, 1654493 average bytes",
        "archived rollout files": "0 files, 0 total bytes, 0 average bytes",
        "goals DB": "/Users/REDACTED/.codex/goals_1.sqlite (file)",
        "goals DB integrity": "ok",
        "log DB": "/Users/REDACTED/.codex/logs_2.sqlite (file)",
        "log DB integrity": "ok",
        "log dir": "/Users/REDACTED/.codex/log (dir)",
        "memories DB": "/Users/REDACTED/.codex/memories_1.sqlite (file)",
        "memories DB integrity": "ok",
        "queue DB": "/Users/REDACTED/.codex/queue_1.sqlite (file)",
        "queue DB integrity": "ok",
        "sqlite home": "/Users/REDACTED/.codex (dir)",
        "state DB": "/Users/REDACTED/.codex/state_5.sqlite (file)",
        "state DB integrity": "ok",
        "thread history DB": "/Users/REDACTED/.codex/thread_history_1.sqlite (file)",
        "thread history DB integrity": "ok"
      },
      "remediation": null,
      "durationMs": 3540
    },
    "state.rollout_db_parity": {
      "id": "state.rollout_db_parity",
      "category": "threads",
      "status": "ok",
      "summary": "rollout files and state DB thread inventory agree",
      "details": {
        "default model provider": "openai",
        "rollout DB active files": "128",
        "rollout DB active rows": "128",
        "rollout DB archive mismatches": "0",
        "rollout DB archived files": "0",
        "rollout DB archived rows": "0",
        "rollout DB duplicate DB paths": "0",
        "rollout DB duplicate rollout thread ids": "0",
        "rollout DB malformed file names": "0",
        "rollout DB missing active rows": "0",
        "rollout DB missing archived rows": "0",
        "rollout DB model providers": "openai=128",
        "rollout DB rows": "128",
        "rollout DB scan cap reached": "false",
        "rollout DB scan errors": "0",
        "rollout DB sources": "exec=47, subagent:thread_spawn=39, cli=18, vscode=17, subagent:other=7",
        "rollout DB stale rows": "0"
      },
      "remediation": null,
      "durationMs": 2253
    },
    "system.environment": {
      "id": "system.environment",
      "category": "system",
      "status": "ok",
      "summary": "OS language en-US",
      "details": {
        "EDITOR": "set",
        "GH_PAGER": "set",
        "GIT_PAGER": "set",
        "LANG": "C.UTF-8",
        "LC_ALL": "C.UTF-8",
        "LC_CTYPE": "C.UTF-8",
        "PAGER": "set",
        "VISUAL": "not set",
        "os": "Mac OS 26.0.1 [64-bit]",
        "os language": "en-US",
        "os type": "Mac OS",
        "os version": "26.0.1"
      },
      "remediation": null,
      "durationMs": 127
    },
    "terminal.env": {
      "id": "terminal.env",
      "category": "terminal",
      "status": "ok",
      "summary": "terminal metadata was detected",
      "details": {
        "COLORTERM": "present",
        "NO_COLOR": "1",
        "TERMINFO": "/Applications/Ghostty.app/Contents/Resources/terminfo (dir)",
        "TERM_PROGRAM": "ghostty",
        "color output": "disabled (NO_COLOR)",
        "effective locale": "C.UTF-8",
        "stderr is terminal": "false",
        "stdin is terminal": "false",
        "stdout is terminal": "false",
        "terminal": "Ghostty",
        "terminal size": "80x24",
        "terminal version": "1.2.0"
      },
      "remediation": null,
      "durationMs": 70
    },
    "terminal.title": {
      "id": "terminal.title",
      "category": "title",
      "status": "ok",
      "summary": "terminal title default",
      "details": {
        "terminal title activity": "true",
        "terminal title items": "activity, project-name",
        "terminal title project source": "git repo root",
        "terminal title project value": "cascade-browser",
        "terminal title source": "default"
      },
      "remediation": null,
      "durationMs": 1
    },
    "updates.status": {
      "id": "updates.status",
      "category": "updates",
      "status": "ok",
      "summary": "update configuration is locally consistent",
      "details": {
        "cached latest version": "0.147.0",
        "check for update on startup": "true",
        "last checked at": "2026-08-12T00:12:12.876114Z",
        "latest version": "0.147.0",
        "latest version status": "current version is not older",
        "npm update target": "/opt/homebrew/lib/node_modules/@openai/codex",
        "update action": "npm install -g @openai/codex",
        "version cache": "/Users/REDACTED/.codex/version.json"
      },
      "remediation": null,
      "durationMs": 1731
    }
  }
}
What issue are you seeing?

A long-running codex exec resume --json session emitted repeated app-server lag errors and dropped hundreds of events. Later attempts to resume the same session could not continue because a custom tool-call result was missing. The process stayed alive and repeatedly printed the same error without producing a response.

What steps can reproduce the bug?
  1. Start or resume a Codex session through codex exec resume --json.
  2. Let the session perform a long-running workflow with repeated tool calls and polling.
  3. Allow the in-process app-server event consumer to fall behind.
  4. Observe event-drop errors.
  5. Resume the same Codex session with another prompt.
What is the expected behavior?

Expected behavior

Event-stream backpressure should not permanently corrupt session state. If events must be dropped, Codex should preserve required tool-call/result pairs, recover the missing state, or terminate the turn with a clear retryable error. A resumed session should not wait indefinitely for an output that can never arrive.

Additional information

Actual behavior

The original run emitted many errors in this form:

in-process app-server event stream lagged; dropped 43 events
in-process app-server event stream lagged; dropped 171 events
in-process app-server event stream lagged; dropped 854 events

Afterward, every resumed run repeatedly emitted:

ERROR codex_core::util: Custom tool call output is missing for call id: call_lgAAyVMavptKZY5NFEwPIkAo

The resumed process produced no assistant response and remained stuck until canceled. Starting additional turns on that same session reproduced the missing-output error.

Impact

  • The session becomes unusable across subsequent resumes.
  • The harness sees a live process but receives no meaningful assistant output.
  • User prompts queue behind or repeatedly re-enter the broken session.
  • Recovery currently requires abandoning or rotating the session rather than simply retrying the turn.

Relevant timing

  • Event-drop burst began: 2026-08-12 17:58:29 UTC
  • First missing tool-output error after resume: 2026-08-12 17:59:53 UTC
  • Error continued across another resumed run through at least 2026-08-12 18:05:56 UTC

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by tracing the app-server event flow and the resumed Codex session path described in the issue. Reproduce a session with dropped app-server events, then verify that resuming it preserves the complete event history without corruption.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
api, backend
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.