[Bug]: Linux — unbounded agent node fleet makes systemd-oomd kill the entire GNOME session, not just Orca (v1.4.188)

### Operating system

Linux

### Orca version

v1.4.188

### Details

**Short summary:** On Ubuntu 24.04 (GNOME/X11), an unbounded fleet of ~20 agent `node` processes (~1.35 GB each, ~27 GB total) exhausts 31 GB RAM + 8 GB swap. The distinctive Linux outcome is the blast radius: `systemd-oomd` responds by killing `user@1000.service/init.scope` — **the entire desktop session**, not just Orca. The user is dumped to the GDM login screen and loses every other app (browser, terminals, editors) along with Orca. This happened **5 times in 37 minutes**.

This is the Linux counterpart of the macOS/Windows "Orca ate my RAM" reports (#13047, #16211, #11218, #10493, #9138). The memory leak itself is already well documented in those; **what is not filed anywhere is that on Linux the standard systemd-oomd configuration converts an Orca memory leak into a full desktop-session kill.** That is the part I think is actionable independently of the leak.

## Host

| | |
| -- | -- |
| OS | Ubuntu 24.04.4 LTS, kernel 6.8.0-138-generic |
| Desktop | GNOME Shell 46.0, X11, GDM |
| systemd | 255 (255.4-1ubuntu8.17) |
| CPU | 12th Gen Intel Core i9-12900H — 14 cores / **20 threads** |
| RAM | 31 GB + 8 GB swap (`/swap.img`) |
| Orca | 1.4.188 (deb, `/opt/Orca`), upgraded from 1.4.185 on Aug 23 |
| Agents | Claude Code |

Note: 1.4.188 ran fine for three days (Aug 23–25, zero oomd kills in those boots). The crashes began Aug 26. So this is triggered by accumulated worker state, not by the version bump itself.

## What happened

Five desktop-session kills, all from `systemd-oomd`:

```
15:24:10 Killed .../user@1000.service/init.scope pressure 64.98% > 50.00% for > 20s
15:44:32 Killed .../user@1000.service/init.scope pressure 64.79% > 50.00% for > 20s
15:44:32 Killed .../user@1000.service/init.scope pressure 57.82% > 50.00% for > 20s
15:55:38 Killed .../user@1000.service/init.scope pressure 64.62% > 50.00% for > 20s
16:00:45 Killed .../user@1000.service/init.scope pressure 61.34% > 50.00% for > 20s
```

Each one logs out the desktop:

```
systemd[1]: user@1000.service: Main process exited, code=killed, status=9/KILL
systemd[1]: user-1000.slice: A process of this unit has been killed by the OOM killer.
```

Confirmed via `journalctl --list-boots` and `last -x`: uptime was continuous 5h44m across all five events. **These are session kills, not reboots** — which is why they are easy to misdiagnose as hardware or thermal faults. From the user's seat it is indistinguishable from the machine dying.

The first Orca run of the day survived ~5 hours (launched 10:20, died 15:23). **Every relaunch afterwards died in 2–8 minutes**, because startup restores the same worker pile. That makes it a soft loop: log in → Orca restores → OOM → logged out → repeat.

## What consumed the memory

Kernel OOM task dump, aggregated by process name. Identical shape at all four kernel OOM events:

| Time | `node` procs | Total RSS |
|---|---:|---:|
| 15:23 | 20 | 26.4 GB |
| 15:43 | 20 | 26.9 GB |
| 15:54 | 20 | 27.0 GB |
| 15:59 | 20 | 27.0 GB |

Individual `node` RSS at the 15:59:56 OOM — a contiguous PID block, i.e. spawned together:

```
419160 1889MB 419217 1752MB 419242 1468MB 419218 1180MB
419216 1882MB 419260 1708MB 419173 1375MB 419225 1135MB
419161 1788MB 419187 1662MB 419192 1348MB 419230 1132MB
419180 1755MB 419241 1548MB 419263 1332MB 419253 1131MB
419154 1299MB 419153 1056MB
419167 1237MB
```

For contrast, Orca's own Electron processes were **tiny** — 11 `orca-ide` processes totalling 0.35 GB. The memory is entirely in the agent CLI children.

Machine state at the OOM:

```
Node 0 Normal free:64520kB min:64564kB <- below the min watermark
Free swap = 172kB
Total swap = 8388604kB <- 8 GB swap 100% consumed
active_anon:16654824kB inactive_anon:12947420kB
pagetables:1057128kB
```

Swap being fully consumed is also the answer to the "CPU is maxed out" symptom: `kswapd0` thrashes and load average hit 12.7. The machine is not computing, it is paging.

## Worth noting: 20 workers on a 20-thread CPU

`nproc` = 20. Node worker count = 20, at every single OOM event. That correlation is exact across four independent events, which suggests concurrency is being derived from core count with no memory dimension. One-worker-per-thread is a reasonable default for CPU-bound workers; for ~1.35 GB agent CLIs it means the memory ceiling scales with core count, so **buying a bigger CPU makes this worse, not better.** A 20-thread machine needs ~27 GB just to hold the fleet.

I want to be careful not to overclaim here — I could not run `orca orchestration worker-list` to confirm the derivation, for the reason below.

## Startup restore is what makes it a loop

Every Orca start today logged this, 8 times total, and **zero times in the two previous boots**:

```
[orchestration] legacy worker provider-ready recovery failed Error: terminal_liveness_unavailable
at OrcaRuntimeService.refreshRestoredOrchestrationAuthority (out/main/index.js:213292:156)
at async Object.reconcile (out/main/index.js:265250:3)
at async recoverLegacyWorkerTerminalsForRendererStartup (out/main/index.js:240318:3)
```

`recoverLegacyWorkerTerminalsForRendererStartup` + the "legacy" naming lines up with #13047, where completed workers sit at `terminalState: retained` / `retainedReason: legacy_ambiguous` and keep their agent CLI alive. Here the recovery path is also *erroring* and retrying, and the fleet is rebuilt on each launch — which is why the time-to-death dropped from 5 hours to 2–8 minutes.

I could not confirm the `retained` states directly with `orca orchestration worker-list --json`, because Orca cannot stay alive long enough to query. If there is a way to read that state offline from `~/.config/orca`, I am happy to pull it.

## The daemon records almost none of this

Final daemon run (`daemon-v36`, pid 414342, started 15:57), complete event census from `~/.config/orca/logs/daemon.log`:

```
1 startup
1 ready
4 client-hello-accepted
2 session-created <- two sessions
```

Then the log **stops entirely** at `2026-08-26T12:57:18.484Z`. The OOM was 2.5 minutes later at 12:59:56Z. Nothing in between.

So: **2 `session-created` events, 20 node processes, 27 GB — and the daemon logged nothing about the other 18 or about the memory growth.** The main process also logged `[claude-live-pty] Seeded 2 persisted Claude session id(s)`. Two sessions seeded, twenty processes spawned.

This matches #11218 ("daemon went silent ~9 minutes before the freeze... No OOM, no error, no warning — it just stops emitting") and #16211's finding that diagnostics report working-set only. There is no telemetry that would let a user self-diagnose this; I only found it by reading kernel OOM dumps.

## Why the Linux blast radius is worse — the part I think is new

Two independent mechanisms fire, and both pick the wrong victim.

**1. `systemd-oomd` kills the whole session.** The limit comes from a **stock Ubuntu file**, not user configuration:

```
$ cat /usr/lib/systemd/system/user@.service.d/10-oomd-user-service-defaults.conf
[Service]
ManagedOOMMemoryPressure=kill
ManagedOOMMemoryPressureLimit=50%
```

Because Orca's agent children live under `user@1000.service`, Orca's memory growth is charged to the slice that contains the whole GNOME session. oomd's remedy is to kill `init.scope` — the user's systemd manager — so GNOME Shell, Xorg, and every unrelated app die with it. **Every default-configured Ubuntu desktop will behave this way.** No Orca setting changes it.

**2. The kernel OOM killer targets Orca's small processes.** Orca's renderers carry `oom_score_adj:300`, so the kernel kills them first — but they are only ~130 MB, so it frees nothing and keeps going. Collateral from these five events included `chrome`, `Docker Desktop`, `snap-store`, `tracker-miner-fs`, `wireplumber`, `gnome-terminal-server`, `evolution-alarm`, `ibus-extension`, and `xdg-desktop-portal-gnome`:

```
Out of memory: Killed process 7303 (orca-ide) anon-rss:279008kB oom_score_adj:300
Out of memory: Killed process 389858 (wireplumber) anon-rss:6120kB oom_score_adj:200
Out of memory: Killed process 413447 (gnome-terminal-) anon-rss:10060kB oom_score_adj:200
```

The 1.35 GB `node` agents have no `oom_score_adj` bump and are killed **last**, after the desktop is already gone. The scoring is inverted relative to where the memory actually is.

## Also present: accumulating daemon generations (#9138)

Each version bump left a stale socket behind:

```
$ ls -la ~/.config/orca/daemon/
daemon-v32.sock Aug 16 17:38
daemon-v33.sock Aug 19 09:56
daemon-v34.sock Aug 23 08:11
daemon-v36.sock Aug 26 15:57
daemon-v36.pid (pid 414342 — dead, killed in the OOM)
```

Mentioning this because it makes **downgrading a poor workaround**: rolling back to 1.4.185 would introduce another daemon generation rather than removing one, and per #9138 old generations keep invisible agent sessions alive.

## Reproduction

1. Ubuntu 24.04, GNOME/X11, default `systemd-oomd` (no config changes needed — this is stock).
2. Run an `/orchestration` flow that starts multiple supervised workers; let them complete.
3. Restart Orca. Startup restore rebuilds the finished workers (`recoverLegacyWorkerTerminalsForRendererStartup`).
4. Within 2–8 minutes: RAM + swap exhausted, and `systemd-oomd` kills `user@1000.service/init.scope`. You are logged out to GDM.

Machines with more cores should reproduce faster, since the fleet size tracks thread count.

## What I would expect instead

Separating the leak (already covered by #13047 / #16211) from the Linux-specific parts:

**Linux-specific, and the reason I am filing:**
1. **Do not let agent children take the session down with them.** Spawn the agent fleet into its own cgroup/slice outside `user@.service` — or ship a systemd slice with a `MemoryMax` — so oomd's remedy is scoped to Orca rather than to the user's whole desktop. A `ManagedOOMPreference=avoid` on Orca's own session-critical units, plus an `oom_score_adj` bump on the *agent* processes rather than the renderers, would also stop the kernel from eating GNOME first.
2. **Bound the fleet by memory, not by `nproc`.** Cap concurrent workers by available RAM. On a 20-thread / 31 GB host, one worker per thread cannot fit.
3. **Do not silently rebuild a large fleet at startup.** When restore would exceed available memory, restore lazily or prompt. The 5-hours-to-2-minutes regression is entirely from restore.

**Diagnosability (cross-platform, but it is what made this a multi-hour investigation):**
4. Log the fleet size and per-child RSS when spawning workers. Right now `2 session-created` is the only trace of a 27 GB, 20-process fleet, and the daemon goes silent before dying.
5. `orca diagnostics memory` should report RSS/private bytes, not working-set (per #16211), and should include the agent CLI children.

Happy to supply full `journalctl -k` OOM dumps, the complete `daemon.log`, or run any diagnostic that survives a short-lived process. I have the raw journal for all five events.

## Related

- #13047 — retained `legacy_ambiguous` workers keep agent CLIs alive (the leak this triggers on)
- #16211 — same version 1.4.188, Windows, 32 retained workers / ~40 GB, working-set under-reporting
- #11218 — macOS, memory runaway + hard freeze, daemon silent before the event
- #10493 — macOS, unbounded agent process growth panics the kernel; restore makes it self-perpetuating
- #9138 — app updates leave previous daemon generations running
- #16084 — one OOM-killed renderer takes down the whole `orca serve` runtime