openai / openai/codex

Windows: startup livelock (endless workspace-root directory walk) when a permission profile has any * deny rule; TUI never leaves "model: loading" (0.148.0-0.151.0-alpha.6)

Open
#41,257 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug CLI config performance windows-os
Dominant language
Rust
Stars
125k
Forks
19.4k
PR merge metrics
PR metrics pending

Description

Environment

  • Windows 11 Pro build 26200, PowerShell 7.6.4
  • @openai/codex via npm; reproduced on 0.148.0, 0.149.0, 0.150.1 and 0.151.0-alpha.6; 0.147.0 is clean with the identical config (it ran the exact same configuration for six days before the update)
  • [windows] sandbox = "elevated", permission profiles configured under [permissions.*]

Symptom

The TUI prints model: loading and never leaves it ("Model selection is disabled until startup completes"); typed input queues and is never processed; codex exec produces no output. The process burns ~2.6-2.8 cores indefinitely (observed past 6,400 CPU-seconds on one wedged PID).

Why this is invisible to normal profiling (the measurement trap)

On a hung process, ReadOperationCount, WriteOperationCount, ReadTransferCount and WriteTransferCount are all frozen, so every ordinary tool concludes "pure spin loop, no I/O". The entire signal lives in OtherOperationCount, climbing at ~100,000-148,000 ops/sec: Windows directory enumeration (NtCreateFile + NtQueryDirectoryFileEx + NtClose) moves zero read/write bytes and lands entirely in that counter. This is very likely why the bug has few reports despite being a hard hang.

Thread-stack evidence

Sampled via GetThreadContext + dbghelp export resolution on a hung 0.151.0-alpha.6 process (identical signature on 0.150.1):

  • exactly 3 hot threads at ~equal CPU, all looping through ntdll!NtCreateFile+0x14, ntdll!ZwQueryDirectoryFileEx+0x14, ntdll!NtQueryInformationFile+0x14, ntdll!ZwClose+0x14, entered via KERNELBASE!CreateFileW from codex.exe code
  • a 4th thread parked in ZwWaitForAlertByThreadId (the UI, waiting forever)
  • handle sampling shows 3 FILE_LIST_DIRECTORY handles (access 0x100001) descending the same tree in lockstep
  • earlier forensics on a wedged PID recorded the walk wrapping around and re-walking the same root 22 times in 29.7 minutes (~80 s per pass over a tree that .NET Directory.GetDirectories(..., AllDirectories) enumerates in 1.15 s, i.e. roughly 70x overhead per pass, on top of never terminating)

What is walked

The selected (effective) permission profile's workspace_roots. Profiles that are defined but not selected do not trigger it. cwd does not matter: reproduced from an empty scratch directory.

Trigger isolation (scratch CODEX_HOME bisect, real TUI, real auth, one variable per arm)

Config arm (profile selected via -c default_permissions=...) Result
profile with a large workspace root, zero filesystem deny rules clean start, ~3 s CPU total, flat afterward
same + exact-path deny rules only (e.g. ".env" = "deny") clean start under the elevated backend
same + any deny rule whose key contains * (a single "*.env" = "deny" suffices; "*/..." enumerated forms and "**/..." forms behave the same) infinite walk, hang
same starred rules but glob_scan_max_depth = 2 still hangs identically (the knob does not bound this walk)

Version boundary with config held constant: 0.147.0 clean; 0.148.0+ hang.

Note: -c 'permissions.<name>.workspace_roots={}' on the command line is a silent no-op (table overrides merge rather than replace), which makes this easy to misdiagnose as config-independent.

Minimal repro

[permissions.repro]
extends = ":read-only"

[permissions.repro.workspace_roots]
'C:\some\large\tree' = true          # anything with a few thousand directories

[permissions.repro.filesystem]
glob_scan_max_depth = 16

[permissions.repro.filesystem.":workspace_roots"]
"." = "read"
"*.env" = "deny"                     # any one starred deny rule suffices

codex -c default_permissions=repro from any cwd hangs at model: loading forever. Remove the one starred line and it starts in seconds.

Workaround

Delete every deny rule containing * from the profiles. This config carried 402 such lines (a 186-rule filesystem policy enumerated to 12 depth levels, an old workaround for #22179-style glob-deny non-enforcement); after removing them, 0.150.1 starts in seconds and completes codex exec turns normally.

Related findings from the same investigation

  1. os error 206 on writer profiles at session init: with the walk fixed, :workspace (write-capable) profiles on a long-established real CODEX_HOME fail startup deterministically with failed to load AGENTS.md instructions for environment 'local': The filename or extension is too long. (os error 206). This looks like the same class as #38985 (sandbox helper payload on the CreateProcess command line), but from a different payload contributor: this profile root has only 169 top-level items (far below the ~500-700 boundary reported there), while the home's cap_sid has accumulated 90 per-cwd/per-root sandbox SIDs (8,938 bytes) over 5 months. A byte-identical robocopy clone of the same CODEX_HOME under another path starts fine, but the clone silently runs a reduced sandbox flow (its .sandbox-bin never receives a command-runner), masking the failure. Read-only profiles on the same home are unaffected. Also present on 0.151.0-alpha.6; worked on 0.147.0 with the same accumulated state. I can file this separately with full logs if useful.
  2. Diagnosability: a hung or failed 0.150.1 writes no rows to logs_2.sqlite and creates no codex-tui.log, even though wedged processes demonstrably reach sqlx migration v51 and fetch models_cache.json. Everything above had to be diagnosed with process counters, thread stacks and handle sampling.

Possibly related existing issues: #37390 (fatal unreadable-glob ripgrep scan of workspace roots on WSL2), #22179 (deny-globs not enforced on reads), #38985 (sandbox setup payload exceeds CreateProcess limit, os error 206).

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the minimal Windows configuration using a selected workspace root and one starred filesystem deny rule, comparing startup on 0.147.0 and 0.148.0+. Trace the selected profile's workspace_roots handling and directory enumeration; the fix is complete when the TUI and codex exec start normally without an endless walk, while exact-path denies and bounded scans remain unaffected.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
cli, operating-systems, security
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.