code-yeongyu / code-yeongyu/senpi

/compact grows to ~28 GB on a 16 GB machine and wedges the process (SIGTERM ineffective)

Open
#1,461 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
TypeScript
Stars
429
Forks
98
Avg merge
5h 3m
Merged PRs (30d)
526

Description

### What happened?

Ran `/compact` on a long session and the process grew to a **28 GB memory footprint on a 16 GB machine**, pushed swap to 93% (9,485 / 10,240 MB), and entered uninterruptible wait (`top` state `stuck`). The TUI stopped accepting input. `SIGTERM` was never delivered; only `SIGKILL` recovered the machine — swap immediately fell to 1,790 MB and free memory went 34% -> 83%.

It is **not** a network hang: `lsof -a -p -i` showed **no outbound socket** for the whole 12 minutes, and there was no log output after the first ~4 minutes.

Session that triggered it: 8,591,529 bytes / 1,709 lines, **1,221 `message` entries** (97.9% of bytes), **3 pre-existing `compaction` entries**. Compaction had succeeded on this same session 3 times before, so this looks like a scale threshold rather than a categorically broken path.

Arithmetic that matches the observation (hypothesis, not confirmed in code):

```
8.6 MB x ~2.5 (JSON -> JS object overhead) ~= 21 MB
21 MB x 1,221 messages ~= 26 GB (observed: 28 G, still growing)
```

That is the shape of retaining a full copy of the history **per message** rather than streaming it. A second candidate is `restoration-tracker` re-expanding previously compacted items — this session carried 3 of them.

### Steps to reproduce

1. Grow a session to ~1,200+ messages / ~8.6 MB jsonl with at least one prior `compaction` entry.
2. Select a large-context model (here `github-copilot/gpt-6-astra`, contextWindow 1,050,000 / maxTokens 128,000).
3. Run `/compact`.
4. Watch `top -l 1 -pid -stats mem,state` — footprint climbs monotonically past physical RAM.

**This also fires unattended.** With `SPECULATIVE_FRACTION = 0.75` and the adaptive threshold ratio clamped to `[0.40, 0.85]`, speculative compaction on a 1,050,000 window triggers at **315,000–669,375 tokens**. Live context was **267,216** — a few turns below the lower bound. So the same blow-up would have happened without anyone typing `/compact`, which is the worse failure mode.

Workaround: all three automatic entry points must be off; disabling only `enabled` leaves the speculative path armed.

```jsonc
"compaction": { "enabled": false, "speculativeEnabled": false, "idleCompactionEnabled": false }
```

### Expected behavior

Compaction peak memory bounded by O(history bytes), not O(history bytes x message count) — and in any case a failure should not be able to exhaust system swap.

Suggested regression test, since output-correctness assertions would not have caught this:

> Compact synthetic sessions of N in {100, 500, 1500} and assert peak RSS < k * (total bytes + largest message) for a small constant k.

A quadratic retention pattern passes N=100 and fails N=1500, which is why existing coverage misses it.

### Version

senpi `2026.9.7` (via `omo-ai` 5.0.0-0.beta.46) - Bun 1.4.2 - macOS 15.6 (Darwin 25.6.0) arm64, 16 GB RAM

---

**On the `-ne` precondition — stating this honestly rather than claiming I met it.**

I did not re-run with `-ne`, because reproducing means deliberately re-entering a state that took a 16 GB machine down. Two pieces of narrowing evidence instead:

- The loaded extension touches compaction only as a **consumer** — an `on_compaction` trigger and a `"compacting"` delivery-buffer state. It does not build the compaction payload; that is core (`context-reduction` / `context-pipeline`).
- No outbound socket during the entire incident argues against extension-spawned work being responsible.

I cannot fully exclude the extension on that basis. If you want the `-ne` run, say so and I will do it with a memory watchdog and report back either way.

Also checked for duplicates before filing: #1340 (closed `completed` 2026-09-05, still reproduces 09-07), #1259 / #1263 / #1388 / #1430 (four open PRs on the admission budget), #1228 (bounded cursor caches — adjacent class, different module). I found no existing memory/OOM report against compaction in the top search results, but I did not review all 136 open issues.

Contributor guide

Open the contributing guide

Research direction

Start at the /compact entry point and trace the core context-reduction and context-pipeline paths, then inspect restoration-tracker and the speculative, idle, and enabled compaction paths. Reproduce with synthetic sessions of 100, 500, and 1,500 messages while monitoring peak RSS; done means compaction no longer scales as history bytes multiplied by message count.

Written by the indexing model from the issue text.

Assessment

Tech stack
bun, typescript
Domain
cli, performance, tooling
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.