code-yeongyu / code-yeongyu/senpi
/compact grows to ~28 GB on a 16 GB machine and wedges the process (SIGTERM ineffective)
- Dominant language
- TypeScript
- Stars
- 429
- Forks
- 98
- Avg merge
- 5h 3m
- Merged PRs (30d)
- 526
Description
### What happened?
Ran `/compact` on a long session and the process grew to a **28 GB memory footprint on a 16 GB machine**, pushed swap to 93% (9,485 / 10,240 MB), and entered uninterruptible wait (`top` state `stuck`). The TUI stopped accepting input. `SIGTERM` was never delivered; only `SIGKILL` recovered the machine — swap immediately fell to 1,790 MB and free memory went 34% -> 83%.
It is **not** a network hang: `lsof -a -p -i` showed **no outbound socket** for the whole 12 minutes, and there was no log output after the first ~4 minutes.
Session that triggered it: 8,591,529 bytes / 1,709 lines, **1,221 `message` entries** (97.9% of bytes), **3 pre-existing `compaction` entries**. Compaction had succeeded on this same session 3 times before, so this looks like a scale threshold rather than a categorically broken path.
Arithmetic that matches the observation (hypothesis, not confirmed in code):
```
8.6 MB x ~2.5 (JSON -> JS object overhead) ~= 21 MB
21 MB x 1,221 messages ~= 26 GB (observed: 28 G, still growing)
```
That is the shape of retaining a full copy of the history **per message** rather than streaming it. A second candidate is `restoration-tracker` re-expanding previously compacted items — this session carried 3 of them.
### Steps to reproduce
1. Grow a session to ~1,200+ messages / ~8.6 MB jsonl with at least one prior `compaction` entry.
2. Select a large-context model (here `github-copilot/gpt-6-astra`, contextWindow 1,050,000 / maxTokens 128,000).
3. Run `/compact`.
4. Watch `top -l 1 -pid -stats mem,state` — footprint climbs monotonically past physical RAM.
**This also fires unattended.** With `SPECULATIVE_FRACTION = 0.75` and the adaptive threshold ratio clamped to `[0.40, 0.85]`, speculative compaction on a 1,050,000 window triggers at **315,000–669,375 tokens**. Live context was **267,216** — a few turns below the lower bound. So the same blow-up would have happened without anyone typing `/compact`, which is the worse failure mode.
Workaround: all three automatic entry points must be off; disabling only `enabled` leaves the speculative path armed.
```jsonc
"compaction": { "enabled": false, "speculativeEnabled": false, "idleCompactionEnabled": false }
```
### Expected behavior
Compaction peak memory bounded by O(history bytes), not O(history bytes x message count) — and in any case a failure should not be able to exhaust system swap.
Suggested regression test, since output-correctness assertions would not have caught this:
> Compact synthetic sessions of N in {100, 500, 1500} and assert peak RSS < k * (total bytes + largest message) for a small constant k.
A quadratic retention pattern passes N=100 and fails N=1500, which is why existing coverage misses it.
### Version
senpi `2026.9.7` (via `omo-ai` 5.0.0-0.beta.46) - Bun 1.4.2 - macOS 15.6 (Darwin 25.6.0) arm64, 16 GB RAM
---
**On the `-ne` precondition — stating this honestly rather than claiming I met it.**
I did not re-run with `-ne`, because reproducing means deliberately re-entering a state that took a 16 GB machine down. Two pieces of narrowing evidence instead:
- The loaded extension touches compaction only as a **consumer** — an `on_compaction` trigger and a `"compacting"` delivery-buffer state. It does not build the compaction payload; that is core (`context-reduction` / `context-pipeline`).
- No outbound socket during the entire incident argues against extension-spawned work being responsible.
I cannot fully exclude the extension on that basis. If you want the `-ne` run, say so and I will do it with a memory watchdog and report back either way.
Also checked for duplicates before filing: #1340 (closed `completed` 2026-09-05, still reproduces 09-07), #1259 / #1263 / #1388 / #1430 (four open PRs on the admission budget), #1228 (bounded cursor caches — adjacent class, different module). I found no existing memory/OOM report against compaction in the top search results, but I did not review all 136 open issues.
Contributor guide
Research direction
Start at the /compact entry point and trace the core context-reduction and context-pipeline paths, then inspect restoration-tracker and the speculative, idle, and enabled compaction paths. Reproduce with synthetic sessions of 100, 500, and 1,500 messages while monitoring peak RSS; done means compaction no longer scales as history bytes multiplied by message count.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- bun, typescript
- Domain
- cli, performance, tooling
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100