openai / openai/codex-plugin-cc

Test suite is not hermetic inside a live Claude Code session: 4 failures at HEAD + fixture state leaks into the user's real plugin data dir

Open
#455 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
JavaScript
Stars
33.3k
Forks
2.3k
PR merge metrics
No merged PRs in 30d

Description

Symptom

Running npm test from a shell inside a live Claude Code session (the natural environment when Claude itself works on this repo):

  1. 4 tests fail at HEAD with no code change:
    • status shows phases, hints, and the latest finished job
    • status preserves adversarial review kind labels
    • result returns the stored output for the latest finished job by default
    • resolveStateDir uses a temp-backed per-workspace directory
  2. Fixture state silently pollutes the user's real plugin data. Two full-suite runs left ~150 fake codex-plugin-test-* workspace dirs and ~420 fake job records under $CLAUDE_PLUGIN_DATA/state/ — indistinguishable from real workspaces in the on-disk registry (fake jobs like "Investigate flaky test", "check failed auth" sit next to real project jobs).

The same suite passes 91/91 under env -i PATH=... HOME=... npm test.

Root cause — two inherited variables

  • CLAUDE_PLUGIN_DATA: resolveStateDir honors it (lib/state.mjs), and a live session points it at the real data dir. Both in-process state writes in tests and spawned companions (buildEnv spreads process.env) then write fixture state there instead of the os.tmpdir() fallback — which is also exactly why the resolveStateDir fallback-path test fails.
  • CODEX_COMPANION_SESSION_ID: set by the plugin's own session-lifecycle hook, inherited by the suite. filterJobsForCurrentSession then hides fixture-seeded jobs that carry no sessionId, so the status/result tests render empty output.

Note the second one is a test-visible instance of a runtime sharp edge: jobs without a matching sessionId are invisible to status even with --all (buildStatusSnapshot applies the session filter before options.all is consulted) — filed separately if useful.

Fix

PR incoming: pin CLAUDE_PLUGIN_DATA to a fresh temp root per test process in tests/helpers.mjs, clear CODEX_COMPANION_SESSION_ID there (session-scoped tests already compose it explicitly), and make the fallback-path state test manage the variable locally — mirroring what its CLAUDE_PLUGIN_DATA sibling test already does.

Validated in the reproducing environment (14 live CLAUDE*/CODEX* vars): 87/91 → 91/91, zero new entries in the real plugin state dir.

🤖 Generated with Claude Code

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with tests/helpers.mjs and the fallback-path state test, then inspect lib/state.mjs and the test environment setup around buildEnv. Run npm test from inside a live Claude Code session and verify all 91 tests pass without creating entries under the real plugin data directory.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript
Domain
testing-qa
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
76/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.