[Bug]: Desktop backend still SIGABRTs with V8 OOM after 9–18h on 0.0.35 (hydration bounds already present)
Nobody has claimed this yet.
- Dominant language
- TypeScript
- Stars
- 23k
- Forks
- 5.9k
- Avg merge
- 11h 14m
- Merged PRs (30d)
- 357
Description
Before submitting
- I searched existing issues and did not find a duplicate.
- I included enough detail to reproduce or investigate the problem.
Area
apps/server
Steps to reproduce
Not a minimal synthetic repro. This is a production desktop profile that has aborted the same way for three weeks.
- Use T3 Code desktop against a local profile whose
~/.t3/userdata/state.sqlitehas grown to ~1 GB, dominated by one long agent thread with tens of thousands oftool.completed/context-window.updatedrows. - Launch T3 Code (Alpha) 0.0.35 and leave the embedded backend up (do not need to keep generating new orchestration events).
- After ~9–18 hours, the backend child aborts. The Electron window stays up and the UI reports a connection / environment-setup failure.
This profile has produced the same abort on 7 Aug, 8 Aug, 16 Aug (twice), 17 Aug, and 29 Aug. Representative runtimes from Node GC banners: 9 h, 11.5 h, 16.3 h, 18 h.
Confirmed not the pre-#6153 “open the huge thread and decode every payload” path: the installed 0.0.35 app.asar already contains THREAD_DETAIL_ACTIVITY_LIMIT = 500, getThreadDetailByIdBounded, listPinnedThreadActivityRowsByThread, and THREAD_RESUME_MAX_GAP = 1000. Newest 500 activities on the offending thread are only ~1.0 MB of JSON. The 29 Aug abort happened with zero new orchestration_events after 27 Aug.
Expected behavior
The desktop backend child should stay within a bounded V8 heap on a large-but-idle profile. Loading a thread should not materialize its complete activity / event history. If the heap is exhausted, the process should fail the request and recover, not abort().
Closed as fixed, but still dying on 0.0.35 with those patches in the binary:
- #5351 / #6153 (500-row thread-detail hydration)
- #5147 (bounded
subscribeThreadreplay + lightweight snapshot HTTP) - #4597 (headless OOM after ~23.5 h, closed as addressed by #5147)
- #996 (heavy-thread backend OOM)
Related but not the same report:
- #4178 (in-memory command model O(N) over all threads ever; in progress; observed as ~1 GB RSS lag, not this SIGABRT)
- #7075 / #6400 (Codex
thread/resumewithoutexcludeTurns; 0.0.35 asar has noexcludeTurnsstring) - #5550 (no supported SQLite retention/compaction)
Actual behavior
The Electron Node backend child (ELECTRON_RUN_AS_NODE=1, ElectronInitializeICUandStartNode), not the renderer, hits V8’s ~3.5–4 GB old-space ceiling and aborts:
Exception Type: EXC_CRASH (SIGABRT)
Termination Reason: SIGNAL, Code 6 Abort trap: 6
Application Specific Information: abort() called
Thread 0: node::OOMErrorHandler → … → v8::Function::Call
→ node::InternalMakeCallback → node::Environment::CheckImmediate
→ uv_run → node::SpinEventLoop → ElectronInitializeICUandStartNode
Parent PID stays alive. Writable regions at death ~4.4 GB on a 16 GB Mac. Historical GC banners from ~/.t3/userdata/logs/server-child.log:
Mark-Compact (reduce) 2756.5 (2860.9) -> 2756.5 (2797.9) MB last resort
FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory
Scavenge 3449.9 (3674.7) -> 3440.6 (3681.7) MB allocation failure
FATAL ERROR: MarkCompactCollector: young object promotion failed
Stripped Electron Framework frames (fontations_ffi$…, V8StackTraceId+40908, etc.) are nearest-symbol noise.
Impact
Major degradation or frequent failure
The window survives so it looks like a network/setup flake (#5351’s misdiagnosis). After ~a day of uptime the local environment dies and the backend restart loop has to rebuild against a 1 GB DB.
Version or commit
T3 Code (Alpha) 0.0.35 (com.t3tools.t3code), Electron Framework 41.5.0, bundled 27 Aug 2026.
Inspected the installed /Applications/T3 Code (Alpha).app/Contents/Resources/app.asar:
| Symbol | Present in 0.0.35? |
|---|---|
THREAD_DETAIL_ACTIVITY_LIMIT = 500 |
yes |
getThreadDetailByIdBounded |
yes |
THREAD_RESUME_MAX_GAP = 1000 |
yes |
projector slice(-500) |
yes |
excludeTurns |
no |
--max-old-space-size |
no |
getSnapshot → listThreadActivityRows(void 0) (unbounded, all threads × all payloads) |
still present |
Environment
- macOS 26.5.1 (25F80), Mac16,1 (Apple M4), 16 GB
- Desktop app, local backend child (node-pty + ffi-rs/fff + msgpackr-extract loaded)
~/.t3/userdata/state.sqlite1.0 GB
| Store | On disk | Rows |
|---|---|---|
orchestration_events |
496 MB | 162,289 |
projection_thread_activities |
367 MB | 94,423 |
| threads | — | 14 |
| messages | — | 4,145 |
Worst thread, still unsettled:
- label: Thread A (long-running agent thread)
- id: redacted
- 83,714 activities, 273 MB
payload_json tool.completed24,210 rows / 258 MB (avg ~10 KB, max 1.0 MB)context-window.updated30,353 rows / 9.9 MB- created 2026-08-04, last
updated_at2026-08-17 - newest 500 activities: 1.07 MB (so the 500-row cap is working as designed if that path is the one used)
Second: Thread B, 8,578 activities / 44 MB.
29 Aug crash window: backend launched 2026-08-28 18:42, aborted 2026-08-29 12:41 (~18 h). MAX(occurred_at) on orchestration_events is 2026-08-27; no events during the crashing process’s lifetime.
Remaining holes that can still feed this abort on 0.0.35:
ProjectionSnapshotQuery.getSnapshot()still hydrateslistThreadActivityRows(undefined)— 94k payloads / 367 MB JSON, which expands many-fold in V8. Review on #6153 already flagged this as a latent bomb if anything still calls it.- No SQLite retention (#5550). The 500-row read cap does not stop the 273 MB thread from sitting on disk or from being pulled in by any unbounded reader (event log, provider resume, traces).
- Codex resume still lacks
excludeTurns(#7075). - Long-uptime growth (#4178 / #4597) still reaches the hard V8 ceiling and
abort()s instead of shedding state.
Logs or stack traces
Apple crash (truncated to the meaningful frames):
Process: T3 Code (Alpha) [32484]
Parent Process: T3 Code (Alpha) [5953]
Version: 0.0.35
Role: Unspecified
Date/Time: 2026-08-29 12:41:58.7751 +0100
Launch Time: 2026-08-28 18:42:46.0337 +0100
Exception Type: EXC_CRASH (SIGABRT)
Terminating Process: T3 Code (Alpha) [32484]
abort() called
Thread 0 Crashed:: Dispatch queue: com.apple.main-thread
0 libsystem_kernel.dylib __pthread_kill
1 libsystem_pthread.dylib pthread_kill
2 libsystem_c.dylib abort
3 Electron Framework node::OOMErrorHandler(char const*, v8::OOMDetails const&) + 356
…
10 Electron Framework node::Buffer::New(node::Environment*, char*, unsigned long) + 636
21 Electron Framework v8::Function::Call(...)
23 Electron Framework node::InternalMakeCallback(...)
25 Electron Framework node::Environment::CheckImmediate(uv_check_s*)
27 Electron Framework uv_run
29 Electron Framework node::SpinEventLoop(node::Environment*)
31 Electron Framework ElectronInitializeICUandStartNode
32 dyld start
VM: Writable regions Total=4.4G
Native: ffi-rs.darwin-arm64.node, msgpackr-extract, node-pty
Earlier backend stderr (same machine, same abort class):
<--- Last few GCs --->
[88561:0x138005b0000] 41377113 ms: Mark-Compact (reduce) 2756.5 (2860.9) -> 2756.5 (2797.9) MB, pooled: 0.0 MB, 61.38 / 0.00 ms (average mu = 0.704, current mu = 0.071) last resort; GC in old space requested
[88561:0x138005b0000] 41377174 ms: Mark-Compact (reduce) 2756.5 (2797.9) -> 2756.5 (2796.9) MB, pooled: 0.0 MB, 60.35 / 0.00 ms (average mu = 0.550, current mu = 0.000) last resort; GC in old space requested
FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory
1: node::OOMErrorHandler(char const*, v8::OOMDetails const&)
Happy to attach the full .ips privately if useful.
Workaround
- Fully quit and relaunch (backend respawns). Recurs after another 9–18 h on this profile.
- Avoid reopening the 83k-activity thread.
- Raising
--max-old-space-sizeonly delays the abort on 16 GB RAM; the live set is not being reclaimed. - Pruning that thread’s
orchestration_events+projection_thread_activitiesandVACUUM(app quit) is the #996 workaround; not tried yet on this profile.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Trace ProjectionSnapshotQuery.getSnapshot() and its callers, especially the listThreadActivityRows(undefined) path, then compare it with the bounded thread-detail and resume paths named in the report. Reproduce with the described large idle profile and backend GC logs; done means large profiles no longer unboundedly hydrate activity payloads and the backend avoids the reported abort or fails the request recoverably.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- electron, node.js, sqlite, typescript
- Domain
- backend, databases, desktop, performance
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 38/100