Windows sandbox cannot recover when `deny_read_acl_state.json` becomes 22 NUL bytes after a system crash

Open
#34,841 9 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
52/100
Issue type
Bug
Clarity
Clearly specified
Activity status
Active
Tech stack
rust

Research direction

Start in codex-rs/windows-sandbox-rs/src/deny_read_state.rs and reproduce setup with a captured 22-NUL state file. Trace serialization, persistence, and reconciliation before choosing the Windows-safe locking and replacement approach. Done means interrupted writes do not publish malformed state, malformed state is quarantined and reconstructed, and setup reports a targeted diagnostic when recovery cannot continue.

Written by the indexing model from the issue text.

Description

app bug sandbox windows-os
Product and environment
  • Codex desktop app on native Windows
  • Observed app package: OpenAI.Codex_26.715.8383.0_x64
  • Windows 11 25H2, build 26200.8875
  • Sandbox mode: ordinary native-Windows workspace-write execution
  • State path: %USERPROFILE%\.codex\.sandbox\deny_read_acl_state.json
Problem

After a Windows kernel crash, the Codex Windows sandbox can become persistently unusable because deny_read_acl_state.json contains exactly 22 zero bytes rather than JSON. Every ordinary sandbox setup tries to deserialize or reconcile this state and fails before the requested command process starts. Execution outside the sandbox still works because it bypasses this setup path.

This has now occurred three times on the same machine. Renaming the malformed file allows the next ordinary sandbox setup to regenerate valid state immediately, confirming that the requested command and repository are not the cause.

The upstream machine crashes are a separate Microsoft Wof.sys issue. This Codex report is about durability and recovery: an external crash can leave a reconstructible state artifact malformed, and Codex then turns that artifact into a persistent outage for every future sandboxed command.

Observed error and file contents

Ordinary commands fail before PowerShell or the requested process starts:

windows sandbox: helper_unknown_error: apply deny-read ACLs

The malformed file is:

Length: 22 bytes
Hex:    00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00

The intended empty serialized state is also 22 bytes:

{
  "principals": {}
}
Reproduction using captured state
  1. Place a captured 22-NUL deny_read_acl_state.json at the state path.
  2. Run any ordinary sandboxed command from a valid workspace.
  3. Observe that sandbox setup fails before the requested process starts.
  4. Preserve and rename the malformed file.
  5. Run an ordinary sandboxed command again.
  6. Codex regenerates a valid 22-byte JSON file and the command starts successfully.

This recovery has been repeated after multiple crashes.

Evidence from three incidents
Incident State length State content Write-to-reboot interval Result after reboot
1 22 bytes all NUL ~20.6 seconds every ordinary sandbox setup failed
2 22 bytes all NUL ~17.3 seconds same
3 22 bytes all NUL 17.364 seconds same

The first two Windows minidumps independently identify the upstream crashes as bugcheck 0x139/A in Wof!WofPreDirectoryControlCallback+0x695, with the same Microsoft failure hash. The third Windows event is also 0x139/A; its dump has not yet been independently analyzed. Those kernel details establish the external-crash context, but they are not required to reproduce Codex's malformed-state failure.

Relevant implementation

The current Windows deny-read state implementation serializes with serde_json::to_vec_pretty and persists using a direct std::fs::write(path, bytes):

codex-rs/windows-sandbox-rs/src/deny_read_state.rs

The observed implementation does not provide:

  • same-directory temporary-file creation followed by atomic replacement;
  • explicit durable flushing before publication;
  • validation of the newly written file before replacing the prior state;
  • malformed-state quarantine and safe reconstruction;
  • an interprocess lock around state reconciliation.

Regardless of the upstream kernel fault, a malformed reconstructible state file should not permanently disable sandbox setup.

Expected behavior
  • A crash during or shortly after persistence should leave either the previous valid state or a complete new state.
  • If state is malformed, setup should quarantine it for diagnostics, reconstruct safe state, and continue or emit a targeted actionable diagnostic.
  • A corrupt reconstructible state artifact should not prevent every sandboxed command from launching indefinitely.
Requested fix
  1. Serialize and validate the complete next state in memory.
  2. Write it to a unique temporary file in the same directory.
  3. Flush the temporary file durably (sync_all or the appropriate Windows equivalent).
  4. Atomically replace the destination while preserving the previous valid file until publication.
  5. Coordinate writers with an interprocess mutex or equivalent lock.
  6. On deserialization failure, quarantine the malformed file and safely reconstruct state.
  7. Report a specific malformed-state diagnostic rather than the generic ACL-application error.
Privacy

This public report intentionally excludes usernames, repository names and paths, thread IDs, raw dumps, .codex databases, sandbox logs, credentials, and secrets. Sanitized debugger conclusions can be supplied publicly; raw evidence can be offered to OpenAI through a private channel if requested.

Dominant language
Rust
Stars
125k
Forks
19.5k
Avg merge
1m
Merged PRs (30d)
1k

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from openai/codex

All issues in openai/codex

Similar issues

More Rust issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.