pingdotgg / pingdotgg/t3code

[Bug]: Switching Codex provider instances can leave a thread stuck on "already has an active writer"

Open
#8,259 9 comments 7 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
TypeScript
Stars
23k
Forks
5.9k
Avg merge
11h 14m
Merged PRs (30d)
357

Description

Before submitting
  • I searched existing issues and did not find a duplicate.
  • I included enough detail to reproduce or investigate the problem.
Area

apps/server

Steps to reproduce
  1. Configure two enabled Codex provider instances in T3 Code that can access the same Codex thread/session store. Use different model selections so T3 treats them as different instances, for example codex and codex_a2.
  2. Start a thread on instance A and send a message.
  3. Interrupt the turn, but leave T3 Code running.
  4. In the same T3 thread, switch the selected model/provider instance to B and send another message.
  5. Retry the message after the first failure.

The captured repro used T3 thread e76d... and Codex thread 01a03.... Identifiers are abbreviated here.

Expected behavior

T3 should release the old provider session before starting the replacement, or perform the replacement atomically. The Codex app-server should have at most one writer for the thread, and the new turn should start once without making the thread unusable.

Actual behavior

The interrupted turn is persisted as interrupted, but the old Codex app-server runtime remains active. When the provider instance is changed, T3 logs that it is restarting the provider session and passes the previous resume cursor to the new instance.

The new Codex app-server then rejects the request because the same Codex thread already has a writer:

ProviderAdapterProcessError: Provider adapter process error (codex) for thread <t3-thread-id>: thread <codex-thread-id> already has an active writer
  [cause]: CodexAppServerRequestError: thread <codex-thread-id> already has an active writer

In the captured run:

  • the first turn ended at 06:28:52Z with turn/completed: interrupted;
  • at 06:29:16Z T3 changed the selection from codex to codex_a2;
  • the restart log reported currentInstanceId=codex, desiredInstanceId=codex_a2, and hasResumeCursor=true;
  • the provider start failed before a new turn began;
  • a retry at 06:29:52Z failed with the same error.

The current implementation appears to start the target adapter before stopping stale sessions:

Impact

Blocks work completely

Version or commit

T3 Code server package 0.0.34

Environment
  • Linux 6.1.0-40-amd64, x86_64
  • Node.js v24.11.1
  • Codex CLI 0.149.1
  • T3 web/server with two Codex provider instances
Logs or stack traces
2026-08-26T06:29:16Z
provider command reactor restarting provider session
currentInstanceId=codex
desiredInstanceId=codex_a2
hasResumeCursor=true

2026-08-26T06:29:16Z
CodexAppServerRequestError: thread <codex-thread-id> already has an active writer

2026-08-26T06:29:52Z
CodexAppServerRequestError: thread <codex-thread-id> already has an active writer

No raw logs, prompts, paths, or database files are attached because they may contain private source code or other sensitive data.

Workaround

Avoid switching Codex provider instances inside an existing T3 thread. Start a new thread on the target instance instead. Restarting T3 before retrying may release the stale runtime, but it also interrupts other active provider sessions.

Suggested fix
  • Enforce a single-writer handoff when changing provider instances: stop or detach the old session before starting the new one, or use an atomic replacement protocol.
  • If replacement fails, clean up the attempted target session and leave the thread in an actionable recoverable state.
  • Ensure stale lifecycle events from the replaced instance cannot overwrite the new binding.
  • Add a regression test that interrupts a Codex turn, switches from instance A to instance B, and verifies that the next turn starts exactly once without an active-writer error.
Related issues
  • #7075 covers large Codex resume responses that can exhaust the T3 server heap. This report concerns the provider-session writer conflict that occurs during instance replacement.
  • #4944 covers stale provider shutdown during a Claude instance switch. The lifecycle problem is similar, but this report reproduces the Codex active-writer error.
Disclosure

I used AI assistance to organize and cross-check the local T3 logs and source. I reviewed the final report, and identifiers and log paths were redacted.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in apps/server/src/provider/Layers/ProviderService.ts at startSession and stopStaleSessionsForThread, then trace instance switching in apps/server/src/orchestration/Layers/ProviderCommandReactor.ts and session state in apps/server/src/provider/Layers/CodexAdapter.ts. Reproduce the interrupted-turn switch from Codex instance A to B, then add a regression test showing the replacement releases the old writer and the next turn starts exactly once without an active-writer error.

Written by the indexing model from the issue text.

Assessment

Tech stack
typescript
Domain
backend
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.