BOHICA-LABS / BOHICA-LABS/vsdd-factory

feat(context-durability): detect & advise on accelerating compaction cadence (thrash signal) from the flush log

Open
#318 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Rust
Stars
2
Forks
1
Avg merge
6h 43m
Merged PRs (30d)
29

Description

## Summary

Nothing in the factory computes the **interval between consecutive context compactions** or reacts when that interval collapses — the signature of a working set that persistently overflows the context window. The timestamps to detect it already exist (`.factory/hooks/precompact-flush-log` records one timestamped line per compaction), but the log's only consumer is the prune script. This requests deriving compaction **cadence** and surfacing a thrash advisory when compactions accelerate past a threshold.

## Why size-health doesn't cover it

`check-state-health` / `factory-health` measure STATE.md **line count** — a *static* proxy that is orthogonal to compaction *frequency*. A session can thrash on a large spec corpus (re-reading 70 files, compacting every ~2–3 min) while STATE.md stays small and reports HEALTHY throughout. Cadence is the dynamic signal; size is not a substitute.

## Proposed feature

**Signal.** Inter-compaction interval from the last K lines of `.factory/hooks/precompact-flush-log` (single source, already timestamped, no new schema). Where #317 lands, the same signal is available directly off the emitted `context.compaction` event rate.

**Detection (configurable, sensible defaults).**
- Absolute floor — last interval `< T_min` (default ~3 min).
- Rolling rate — `≥ N` compactions within `W` minutes (default ≥3 in 10 min).
- Optionally — monotonically shrinking intervals across the last K events (acceleration, not just frequency).

**Reaction (advisory-first; PostCompact cannot block compaction anyway).**
- Append a cadence line to the **PostCompact re-anchor block** when tripped, e.g.
`⚠ compaction cadence high: 3 in 8m (last interval 2m40s) — working set likely exceeds context window; consider index-first review, sharding the corpus, or killing + re-dispatching the current long-running dispatch.`
- Add a **CADENCE** line (HEALTHY / WARNING / THRASH) to `check-state-health` / `factory-health`, beside the existing STATE.md-size line.
- Generalize #306 mitigation #5: when a *backgrounded* dispatch trips the threshold, recommend kill + re-dispatch with a sharded strategy.

**Suggested home.** Extend `postcompact-reanchor.sh` — it already fires on every compaction, reads `.factory/` logs, and is advisory + fail-open. Computing the interval from the last K flush-log lines and appending one advisory line needs no new hook event and no schema change.

## Constraints (SOUL.md alignment)

- **Advisory / opt-in, never mandatory** — cannot and must not block compaction; default-on is fine (a stdout/log advisory with no side effects).
- **Single source, no drift surface** — derive from the existing flush log (or #317's event); do **not** add a hand-maintained cadence ledger.
- **No phone-home; gradual elaboration** — start with the threshold advisory; defer per-cycle baselines and trend dashboards until a concrete need.

## Relationship

- Builds on #317 (emit compaction events via OTEL) for the Grafana-alertable path; works off the local flush log even without #317.
- Generalizes #306 (per-dispatch thrash, observed as 5 compactions in ~13 min) into a session/pipeline-level self-monitor; #306 becomes the reviewer-agent consumer of this signal.
- A fallback signal source (context-size retrograde proxy, for environments without #317) is covered in the sibling issue linked below.

## Environment

- vsdd-factory develop tip (E-18: S-18.04a/05). Telemetry: `.factory/hooks/precompact-flush-log`, `.factory/logs/postcompact-reanchor-.jsonl`.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.