PaulRBerg / PaulRBerg/agent-toolkit

notify: cleanup exports accumulate forever; retention_days never prunes the exports directory

Open
#1 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Rust
Stars
0
Forks
1
PR merge metrics
No merged PRs in 30d

Description

Problem

cleanup_old_data writes a full-table JSON backup on every run, but nothing ever removes those backups. cleanup.retention_days governs SQLite rows only — it never applies to the exports directory.

notify/src/state/mod.rs#L283-L289:

if export_before {
    let timestamp = Local::now().format("%Y%m%d_%H%M%S");
    let path = export_dir().join(format!("sessions_before_cleanup_{timestamp}.json"));
    stats.rows_exported = self.export_to_json(&path, None); // None = whole table
}

run_scheduled_cleanup fires at most once every 24h, so this writes one new timestamped file per day, indefinitely. There is no remove_file, read_dir, or remove_dir anywhere under notify/src/, so the directory only grows.

Both relevant defaults are on (auto_cleanup_enabled: true, export_before_cleanup: true), so every default install accumulates these files.

Because the retention DELETE works correctly here, each file stays bounded at roughly retention_days of rows — the growth is linear rather than per-file unbounded. But the file count has no ceiling. At my usage (~14 sessions/day, ~765 B/row) a 30-day export is ~330 KB, which is roughly 120 MB/year never reclaimed.

I hit a much more severe version of this on the archived ai-notify 1.0.0 Python release, where the retention DELETE compared the TEXT created_at against an integer Unix timestamp and so never matched a row. With the table never shrinking, each daily export dumped the entire history: 166 files / 216 MB, per-file size growing linearly and total size quadratically. The Rust rewrite already fixed that comparison (datetime('now', ?1)), so this report is only about the exports themselves — but it is what surfaced the missing pruning.

Solution

Cap the exports directory by count: keep the N most recent and delete the rest after a successful export. A count-based cap looks like the better fit here than reusing retention_days:

  • it bounds the directory deterministically regardless of session volume, whereas an age-based rule still grows without limit for a heavy user;
  • AGENTS.md already describes session data as "intentionally transient rather than strictly durable", so a handful of recent snapshots is the useful set;
  • it keeps the newest backup even if a machine sits idle past the retention window.

Suggested shape: a cleanup.export_retention key (default 5), pruned immediately after a successful export so a failed export never deletes the last good one.

PR follows. Happy to rework it if you would prefer age-based pruning, a total-size cap, or a different default.

Environment

  • PaulRBerg/agent-toolkit @ 7bcb560, notify/
  • macOS Tahoe 26.4.1

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in notify/src/state/mod.rs around lines 283-289 and read AGENTS.md for the session-data policy. Trace cleanup_old_data and run_scheduled_cleanup, then identify the configuration path for cleanup settings. Done means successful exports retain only the configured number of newest files, while a failed export does not remove the last good snapshot.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
backend
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
70/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.