hooks pretrain appends duplicate memories on every run (6 -> 12 -> 18) while always reporting "Memories stored: 6"

Summary

agentic-flow hooks pretrain appends a byte-identical copy of every memory to .agentic-flow/intelligence.json on each run, instead of replacing or de-duplicating them. The CLI reports Memories stored: 6 on every run regardless of how many are actually in the store, so the growth is invisible from the output.

patterns, sequences and dirPatterns are keyed objects and de-duplicate correctly. memories is a plain array and is the only unbounded field.

Version: agentic-flow 2.1.2 (also reproduced on 2.1.0), macOS 15 (darwin 25.6.0), Node from a standard npm global install.

Reproduce

rm -f .agentic-flow/intelligence.json

npx agentic-flow hooks pretrain
node -e "console.log(require('./.agentic-flow/intelligence.json').memories.length)"   # 6

npx agentic-flow hooks pretrain
node -e "console.log(require('./.agentic-flow/intelligence.json').memories.length)"   # 12

npx agentic-flow hooks pretrain
node -e "console.log(require('./.agentic-flow/intelligence.json').memories.length)"   # 18

Both runs print the same summary:

📊 Pretrain Complete!
   📁 Files analyzed: 4756
   🧩 Patterns created: 4756
   💾 Memories stored: 6
   🔗 Co-edits found: 100
   ⏱️  Duration: undefinedms

Evidence the appended entries are duplicates, not new data

Same repository, no files changed between runs:

const m = require('./.agentic-flow/intelligence.json').memories;
const c = m.map(x => x.content);

m.length                        // 12
new Set(c).size                 // 6
JSON.stringify(c.slice(0,6)) === JSON.stringify(c.slice(6))   // true
JSON.stringify(m[0].embedding) === JSON.stringify(m[6].embedding) // true
m[0].created                    // 2026-08-03T08:43:56.200Z
m[6].created                    // 2026-08-03T08:44:32.405Z

The pairs differ only in created. Content and embedding vectors are identical.

Store size grows accordingly on identical input:

run	memories	intelligence.json
1 (clean)	6	28,243 bytes
2	12	42,684 bytes

Deleting the file and re-running returns it to exactly 6 / 28,243 bytes, so pretrain itself is deterministic — the accumulation comes purely from appending to the existing store.

Impact

• Unbounded growth: ~14 KB per invocation, forever, for a repo whose real content never changes. pretrain is documented as a step users run to (re)bootstrap, so repeat runs are the expected usage.
• Retrieval skew: duplicates carry identical embedding vectors, so any similarity search over memories returns the same item N times at the same score, crowding out distinct memories in a top-k window.
• Silent: the summary always says Memories stored: 6, which reads as "the store contains 6". Nothing in the CLI output reveals that the store now holds 12, 18, 24…

Suggested fix

De-duplicate on write, keyed on memory content (or a content hash), keeping the newest created. Alternatively, have pretrain replace the memories array rather than concatenating, since it is a full re-scan of the repository rather than an incremental update.

If the append is intentional (e.g. to retain history), the summary line should report the resulting store size rather than the count produced by this run.

Related

• #182 — write commands that report success and persist nothing. This is the inverse failure mode in the same store: a write that persists more than it reports.

Minor, same command

Duration: undefinedms is printed on every run — the duration value is never populated.

Start with the agentic-flow hooks pretrain command and the code that writes .agentic-flow/intelligence.json; reproduce the issue by running pretrain repeatedly and checking memories.length. Decide whether repeated full scans replace or de-duplicate memories, and verify that the reported count matches the resulting store size without duplicate entries.