hiper2d / hiper2d/marlow

Memory design: lossy compaction (prune_recent) vs append-only + supersession?

Open
#3 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
1
Forks
0
PR merge metrics
No merged PRs in 30d

Description

Hi Aliaksei — not a bug, a design question. I found **marlow** while looking for prior art on file-based agent memory, and it's the closest thing to what I'm building, so I read the memory path properly before opening this.

What I'm looking at: `handlers/grade_memory.py` prunes `memory/recent/` via `prune_recent()` → `f.unlink()` (default `keep_days=3`), while the day's ticks get compressed into a one-paragraph rollup in `memory/working.md` inside the Marlow session. So after ~3 days the only surviving trace of a tick is whatever that editorial pass wrote — and per your README that pass "summarizes without judging" and treats all content uniformly.

I'm building the opposite bet: a local-first, model-agnostic memory that is **append-only and never deletes**, and revises by *supersession* (append a new record pointing back at the old one with a reason) precisely so failed runs, rejected ideas and negative results survive as labelled evidence. Doctrine, if useful: https://github.com/krapcys1-maker/adaptive-agent-memory-research/blob/main/docs/00-project/operating-doctrine.md

Where I'd love your take:

1. **The negative case.** When a rollup compresses a day, does a *negative* outcome — "tried X, it broke" / "decided against Y" — reliably survive into `working.md`, or can it drop out because the summarizer didn't judge it worth a line? Have you ever been bitten by losing a "why I did **not** do X" after the 3-day prune?
2. **Recovery.** `working.md` is capped (~10KB) and gets rewritten as older rollups compact. Do you ever need the pre-compaction version back, and is git history on `master` your only recovery path once `recent/` is unlinked?
3. **Where each breaks.** Your bet is bounded-token / low-cost; mine is unbounded-growth / full-provenance. In practice, where do you think each one actually breaks first?

Genuinely curious rather than critiquing — I think we made opposite tradeoffs on the same problem and I want to know where mine is naive. Either way I'll keep watching the repo; `docs/model-findings.md` (the per-provider caching + reasoning-variance notes) was the most useful cross-model write-up I've read in a while.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reading handlers/grade_memory.py, especially prune_recent() and its keep_days=3 behavior, then inspect memory/recent/ and memory/working.md. Review docs/model-findings.md for the project's existing memory and provider findings. The issue has no defined implementation scope or acceptance criteria; a concrete decision about compaction, retention, or supersession would be needed before work is considered done.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
ai
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.