openai / openai/codex

Auto-review can cause runaway quota consumption when Windows sandbox bugs repeatedly escalate workspace-local operations

Open
#39,408 0 comments 2 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug extension rate-limits safety-check sandbox windows-os
Dominant language
Rust
Stars
125k
Forks
19.4k
PR merge metrics
PR metrics pending

Description

What version of the IDE extension are you using?

26.814.41407

What subscription do you have?

ChatGPT Plus

Which IDE are you using?

VS Code

What platform is your computer?

Microsoft Windows NT 10.0.26200.0 x64

What issue are you seeing?

Summary

With "Approve for me" / Auto-review enabled on native Windows, failures in the
Codex sandbox/tooling can turn normal workspace-local development operations
into a very large number of separate codex-auto-review calls.

These reviewer calls appear as separate model turns in Codex analytics and contribute to the user's Codex usage limits.

I am not reporting merely that apply_patch or pytest fail on Windows; those
underlying problems already have related reports. The additional problem is
that a sandbox/tool failure can repeatedly convert operations that should stay
inside the writable workspace into quota-consuming Auto-review requests, with
no apparent circuit breaker or warning to the user.

In one affected day, Aug. 13, my Codex analytics reported:

  • 201 codex-auto-review turns

  • I was able to recover and analyze 198 of those review turns from local rollout
    files

  • 113 of 198 (57.1%) were retries of the built-in apply_patch operation after
    its sandboxed execution failed

  • those 113 patches targeted ordinary files inside the configured writable
    repository

  • none of those 113 targeted .git, .codex, .agents, or paths outside the
    workspace

  • the retry reason repeatedly recorded was:

    command failed; retry without sandbox?

  • 197 of the 198 recovered Auto-review decisions ultimately returned allow

Thus a known failure in a routine built-in editing operation accounted for more
than half of all Auto-review turns that day.

This is reproducible across multiple high-usage days. On Aug. 4, account analytics recorded 299 codex-auto-review turns. Of 250 reviewer turns recovered from local rollout files, 123 (49.2%) were apply_patch retries with the exact reason
command failed; retry without sandbox?. This represents at least 41.1% of all Auto-review turns shown by analytics that day. On Aug. 13, the same failure accounted for 113 of 198 recovered reviewer turns (57.1%). Across those two days alone, 236 Auto-review model calls are directly attributable to this one known Windows apply_patch sandbox/helper failure.

Further analysis of the 250 recovered Aug. 4 Auto-review turns shows that 179
(71.6%) were triggered by operations that were intended to remain entirely
inside the writable workspace: 123 were apply_patch failure/retry reviews and
56 were ordinary project-local shell/build/processing operations that were
forced to require_escalated after Windows sandbox failures. Another 39 reviews
were spent diagnosing or repairing the broken Windows sandbox itself. Including
that troubleshooting fallout, 218 of 250 recovered reviews (87.2%) were directly
caused by, or downstream of, sandbox/tooling failures rather than normal
approval-worthy boundary crossings.

There was a second similar source of review churn involving pytest on Windows.
pytest-created temporary directories can receive restrictive 0o700 Windows
ACLs and become inaccessible to the Codex sandbox, resulting in
PermissionError: [WinError 5] Access is denied. I also observed access failures
in .pytest_cache. On affected later sessions, Codex responded to test/validation
permission failures by retrying commands outside the sandbox, generating
additional Auto-review calls.

The important common pattern is:

  1. Codex performs an operation that should be valid inside the writable workspace.
  2. An internal sandbox/tooling incompatibility causes it to fail.
  3. Codex interprets the failure as requiring execution outside the sandbox.
  4. "Approve for me" sends that retry to codex-auto-review.
  5. The reviewer allows the operation.
  6. The same failure occurs again on the next routine edit/test.
  7. Steps 3-6 can repeat dozens or hundreds of times during one coding session.
  8. These automatic reviewer turns consume the user's plan usage.

This makes an underlying sandbox bug substantially more expensive when
Auto-review is enabled. Without Auto-review, repeated approval prompts would
also make the problem immediately visible to the user. With Auto-review, the
agent can continue generating reviewer calls in the background while ordinary
development work proceeds.

Relative impact on account usage

The Auto-review volume was not a small fraction of overall Codex activity.

On Aug. 4, account analytics reported:

  • 299 codex-auto-review turns
  • 22 Sol turns
  • 1 Luna turn
  • 1 GPT-5.5 turn

That is 323 total recorded model turns, of which approximately 92.6% were
Auto-review turns
.

On Aug. 13, account analytics reported:

  • 201 codex-auto-review turns
  • 12 Sol turns
  • 15 Terra turns
  • 4 Luna turns

That is 232 total recorded model turns, of which approximately 86.6% were
Auto-review turns
.

So on both affected days, the automatic reviewer represented the overwhelming
majority of recorded model turns, rather than incidental approval overhead.

This is especially concerning because a large fraction of those reviewer turns
can be traced to repeated sandbox/helper failures in routine workspace-local
operations. For example, on Aug. 4 at least 123 Auto-review turns were caused
by the known apply_patch failure/retry path, while there were only 24
non-review coding-model turns recorded that entire day.

What steps can reproduce the bug?

The apply_patch path provides the clearest reproduction on the affected
Windows installation.

Environment:

  • Native Windows
  • VS Code Codex extension
  • workspace-write permissions
  • approval policy: on-request
  • approvals reviewer: auto_review / "Approve for me"
  • Windows sandbox: elevated
  1. Open a normal writable Git repository.

  2. Ask Codex to make several ordinary repository-local source edits.

  3. Codex uses its built-in apply_patch operation.

  4. On the affected Windows installation, the generated apply_patch.bat
    resolves the helper to the packaged executable under a path equivalent to:

    C:\Program Files\WindowsApps\OpenAI.Codex_<VERSION>_x64__...\app\resources\codex.exe --codex-run-as-apply-patch

  5. That executable cannot be launched from the sandboxed helper context
    (Access is denied / Windows error 5), even though the workspace itself is
    writable.

  6. Codex records the failure and requests:

    command failed; retry without sandbox?

  7. Because "Approve for me" is enabled, this creates a separate
    codex-auto-review turn.

  8. The reviewer normally returns allow, the operation proceeds via the
    escalated path, and Codex continues working.

  9. Repeat normal coding/editing. Each subsequent apply_patch can follow the
    same failure -> escalation -> Auto-review path.

During one real coding day this resulted in 113 Auto-review turns from
apply_patch alone.

A similar repeated escalation pattern can occur with pytest when Windows
0o700 temporary-directory ACLs make pytest-created directories inaccessible
to the sandbox.

What is the expected behavior?

Routine operations inside a configured writable workspace should not generate
quota-consuming approval reviews because an internal sandbox/helper bug caused
them to fail.

In particular:

  1. Built-in apply_patch edits targeting ordinary files inside the writable
    workspace should execute inside the workspace sandbox.

  2. pytest temporary directories created by a sandboxed process should remain
    accessible to that process for normal test execution.

  3. If an internal sandbox/tooling failure nevertheless causes repeated
    escalations, Auto-review should have protection against runaway repeated
    reviewer calls.

Possible protections could include:

  • detecting repeated identical sandbox-failure -> escalation patterns;
  • a circuit breaker after repeated Auto-review approvals for the same failure
    class;
  • automatically switching to a safe in-workspace fallback for a known broken
    built-in operation;
  • deduplicating repeated equivalent approval reviews;
  • warning the user when abnormal Auto-review volume is accumulating;
  • exposing Auto-review usage separately and clearly before it materially
    consumes the user's plan limits.

I am not suggesting that legitimate sandbox boundary crossings should bypass
review. The issue is specifically repeated escalation caused by failures in
operations that should have remained inside the writable workspace.

Additional information

Related issues

These appear to cover individual pieces of this failure but not the runaway
interaction:

  • #31776 — Windows sandbox apply_patch wrapper points to WindowsApps and fails
    under Auto-review / Approve for me.
  • #13965 — earlier Windows apply_patch WindowsApps / CreateProcessAsUserW
    error 5 report.
  • #19791 — Windows sandbox cannot reopen pytest/pytest-xdist temporary
    directories created with 0o700 permissions.
  • #36733 — Guardian/Auto-review sessions can consume substantial plan quota
    without adequate usage visibility.
  • #29717 — Auto-review calls are separate quota-consuming requests and can fail
    after the account usage limit is reached.

I believe this issue is distinct from those reports because it concerns the
feedback loop between them:

workspace-local operation
    -> sandbox/tool bug
    -> retry outside sandbox
    -> separate Auto-review model call
    -> allow
    -> repeat many times
    -> substantial user quota consumption

Observed quantitative impact

On one affected day:

Metric Count
Auto-review turns shown by account analytics 201
Auto-review turns recovered from local rollout data 198
apply_patch sandbox-failure/retry reviews 113
Percentage attributable to apply_patch retries 57.1%
Recovered reviewer decisions returning allow 197 / 198

The raw rollout files are not attached because they contain unrelated private
conversation/project information. I can provide additional redacted diagnostic
excerpts if maintainers identify specific fields that would help investigate.

No repository name, source code, private document name, username, or full
user-specific path is intentionally included in this report.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the native Windows flow with the built-in apply_patch operation, workspace-write permissions, elevated sandboxing, and Auto-review enabled. Trace the apply_patch.bat helper failure and the resulting retry-without-sandbox request, then compare it with pytest temporary-directory failures. Done should prevent repeated workspace-local failures from causing runaway Auto-review calls while preserving legitimate approval reviews.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust, vscode
Domain
developer-experience, devtools, operating-systems, testing-qa
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.