Login-shell tests rely on non-hermetic startup assumptions
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 125k
- Forks
- 19.4k
- PR merge metrics
- PR metrics pending
Description
What issue are you seeing?
On v0.149.0-alpha.3, several Linux codex-core tests fail when a login shell sources startup configuration that emits diagnostics or attempts filesystem writes unavailable inside the sandbox. The extra startup output becomes part of the captured command result, causing exact-output and remote-compaction-parity assertions to fail.
Affected categories include request-permission tests, remote-compaction-parity tests, and a login-shell test that verifies the shell home.
What steps can reproduce the bug?
- Use a Linux environment with non-trivial login-shell startup configuration.
- Run the affected
codex-coretests under a sandbox whose filesystem permissions differ from the host shell environment. - Observe that startup diagnostics are appended to the command output and cause the assertions to fail.
What is the expected behavior?
Tests that exercise login shells should be hermetic. Their expected command output should not depend on ambient user startup files, host-specific hooks, or whether paths used by those hooks are writable inside the sandbox.
Additional information
The failure exposes several assumptions that do not hold for sandboxed login shells:
- isolating
HOMEdoes not necessarily isolate every startup-file or shell-initialization source; - host startup configuration may run in a filesystem namespace with different permissions;
- command-output and parity tests assume that startup hooks produce no observable output.
Suggested fixes:
- run these tests with a fully controlled temporary home and startup-file set;
- explicitly suppress or override ambient startup hooks while retaining dedicated coverage for login-shell behavior; and
- separate shell diagnostics from the command output captured by the test harness.
A related host-environment test-isolation issue is #38906.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the Linux codex-core request-permission, remote-compaction-parity, and login-shell tests described in the issue, reproducing them with non-trivial startup configuration and sandbox restrictions. Trace how login-shell startup output enters the captured command result. Done means these tests use controlled startup state and no longer depend on host hooks, writable paths, or ambient diagnostics.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- linux, rust, shell
- Domain
- devtools, operating-systems, testing
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100