Memory design: lossy compaction (prune_recent) vs append-only + supersession?
- Dominant language
- Python
- Stars
- 1
- Forks
- 0
- PR merge metrics
- No merged PRs in 30d
Description
Hi Aliaksei — not a bug, a design question. I found **marlow** while looking for prior art on file-based agent memory, and it's the closest thing to what I'm building, so I read the memory path properly before opening this.
What I'm looking at: `handlers/grade_memory.py` prunes `memory/recent/` via `prune_recent()` → `f.unlink()` (default `keep_days=3`), while the day's ticks get compressed into a one-paragraph rollup in `memory/working.md` inside the Marlow session. So after ~3 days the only surviving trace of a tick is whatever that editorial pass wrote — and per your README that pass "summarizes without judging" and treats all content uniformly.
I'm building the opposite bet: a local-first, model-agnostic memory that is **append-only and never deletes**, and revises by *supersession* (append a new record pointing back at the old one with a reason) precisely so failed runs, rejected ideas and negative results survive as labelled evidence. Doctrine, if useful: https://github.com/krapcys1-maker/adaptive-agent-memory-research/blob/main/docs/00-project/operating-doctrine.md
Where I'd love your take:
1. **The negative case.** When a rollup compresses a day, does a *negative* outcome — "tried X, it broke" / "decided against Y" — reliably survive into `working.md`, or can it drop out because the summarizer didn't judge it worth a line? Have you ever been bitten by losing a "why I did **not** do X" after the 3-day prune?
2. **Recovery.** `working.md` is capped (~10KB) and gets rewritten as older rollups compact. Do you ever need the pre-compaction version back, and is git history on `master` your only recovery path once `recent/` is unlinked?
3. **Where each breaks.** Your bet is bounded-token / low-cost; mine is unbounded-growth / full-provenance. In practice, where do you think each one actually breaks first?
Genuinely curious rather than critiquing — I think we made opposite tradeoffs on the same problem and I want to know where mine is naive. Either way I'll keep watching the repo; `docs/model-findings.md` (the per-provider caching + reasoning-variance notes) was the most useful cross-model write-up I've read in a while.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by reading handlers/grade_memory.py, especially prune_recent() and its keep_days=3 behavior, then inspect memory/recent/ and memory/working.md. Review docs/model-findings.md for the project's existing memory and provider findings. The issue has no defined implementation scope or acceptance criteria; a concrete decision about compaction, retention, or supersession would be needed before work is considered done.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- ai
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100