openai / openai/codex

Login-shell tests rely on non-hermetic startup assumptions

Open
#39,681 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug CLI sandbox
Dominant language
Rust
Stars
125k
Forks
19.4k
PR merge metrics
PR metrics pending

Description

What issue are you seeing?

On v0.149.0-alpha.3, several Linux codex-core tests fail when a login shell sources startup configuration that emits diagnostics or attempts filesystem writes unavailable inside the sandbox. The extra startup output becomes part of the captured command result, causing exact-output and remote-compaction-parity assertions to fail.

Affected categories include request-permission tests, remote-compaction-parity tests, and a login-shell test that verifies the shell home.

What steps can reproduce the bug?
  1. Use a Linux environment with non-trivial login-shell startup configuration.
  2. Run the affected codex-core tests under a sandbox whose filesystem permissions differ from the host shell environment.
  3. Observe that startup diagnostics are appended to the command output and cause the assertions to fail.
What is the expected behavior?

Tests that exercise login shells should be hermetic. Their expected command output should not depend on ambient user startup files, host-specific hooks, or whether paths used by those hooks are writable inside the sandbox.

Additional information

The failure exposes several assumptions that do not hold for sandboxed login shells:

  • isolating HOME does not necessarily isolate every startup-file or shell-initialization source;
  • host startup configuration may run in a filesystem namespace with different permissions;
  • command-output and parity tests assume that startup hooks produce no observable output.

Suggested fixes:

  • run these tests with a fully controlled temporary home and startup-file set;
  • explicitly suppress or override ambient startup hooks while retaining dedicated coverage for login-shell behavior; and
  • separate shell diagnostics from the command output captured by the test harness.

A related host-environment test-isolation issue is #38906.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the Linux codex-core request-permission, remote-compaction-parity, and login-shell tests described in the issue, reproducing them with non-trivial startup configuration and sandbox restrictions. Trace how login-shell startup output enters the captured command result. Done means these tests use controlled startup state and no longer depend on host hooks, writable paths, or ambient diagnostics.

Written by the indexing model from the issue text.

Assessment

Tech stack
linux, rust, shell
Domain
devtools, operating-systems, testing
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.