ruvnet / ruvnet/ruflo

Encryption-at-rest (RFE1) silently orphans the memory corpus after the #2786 agentdb-memory.db split

Open
#2,889 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
TypeScript
Stars
72.8k
Forks
8.6k
Avg merge
2d 23h
Merged PRs (30d)
82

Description

Summary

After upgrading from 3.25.6 to 3.33.0 with encryption-at-rest enabled (CLAUDE_FLOW_ENCRYPT_AT_REST=1), MCP memory tools report an essentially empty database while .swarm/memory.db remains intact on disk.

#2786 introduced agentdb-memory.db to isolate AgentDB from potentially-encrypted files. That fix is correct as far as it goes, but @claude-flow/memory lacks the RFE1 decryption handling that @claude-flow/cli already has. So on first start under 3.33.0 it cannot read the encrypted memory.db, initializes a fresh empty agentdb-memory.db beside it, and reports success.

The underlying data is never damaged — but nothing says so, and the error hints that are surfaced recommend memory init, which is destructive against an encrypted store.

This is a follow-on to #2786, not a duplicate of it: #2786 is about initAgentDB() failing silently, this is about the store-split its fix introduced leaving the existing corpus stranded and unreferenced.

Environment
  • Windows 11
  • Node v24.14.1
  • ruflo 3.33.0
  • @claude-flow/cli 3.33.0
  • @claude-flow/memory 3.0.0-alpha.18
  • CLAUDE_FLOW_ENCRYPT_AT_REST=1, CLAUDE_FLOW_CWD (project root redirected)
Steps to reproduce
  1. On 3.25.6 with CLAUDE_FLOW_ENCRYPT_AT_REST=1, populate the memory store across multiple namespaces so .swarm/memory.db contains encrypted entries.

  2. Upgrade to ruflo 3.33.0 / @claude-flow/cli 3.33.0.

  3. Call any MCP memory tool (memory_stats, memory_list, memory_search_unified, agentdb_hierarchical-recall).

  4. memory_stats returns totalEntries: 1 across 1 namespace, while .swarm/memory.db is still 88,813,600 bytes on disk containing 8,923 entries across 33 namespaces.

  5. Confirm the encrypted database and key are both still valid:

    node -e "const { readFileMaybeEncrypted } = require('@claude-flow/cli/fs-secure'); console.log(readFileMaybeEncrypted('.swarm/memory.db').slice(0,16).toString());"
    

    This prints SQLite format 3 and memory_entries contains all 8,923 rows — decryption succeeds and no data was lost.

Expected vs actual
Scenario Expected Actual
MCP tool access after upgrade Read the existing encrypted memory.db, or migrate its records into agentdb-memory.db Ignores the encrypted memory.db, creates an empty agentdb-memory.db, reports success with 1 entry
Graph edge writer init Sniff the RFE1 header before opening with a native driver graph-edge-writer.js getBridgeDb() calls new BetterSqlite3(dbPath) with no RFE1 sniff, throws SQLITE_NOTADB, swallows it in catch { return null }, blames sql.js, and recommends memory init
Why this is easy to miss

@claude-flow/memory creates a clean agentdb-memory.db without throwing or warning, so the memory subsystem reports healthy while operating on an empty store. getAgentDbPath() in memory-bridge.js derives the path via path.join(path.dirname(getDbPath()), 'agentdb-memory.db') under the stated intent that "sql.js keeps memory.db (possibly encrypted); AgentDB owns agentdb-memory.db". But @claude-flow/cli has RFE1 handling in fs-secure.js (readFileMaybeEncrypted) and memory-initializer.js, and @claude-flow/memory has none.

Separately, graph-edge-writer.js swallows SQLITE_NOTADB and emits a message referencing sql.js — which has not been the driver since #2431 — alongside a recommendation to run memory init. A user who sees an empty store and follows that hint will destructively overwrite their encrypted memory.db. This is the most dangerous part of the report: the tool's own error text recommends the one command that causes data loss.

doctor.js already gained an isEncryptedBlob() primitive in #2737, so the detection mechanism exists in-tree — it is just missing from these read paths.

Related: #1989 and #2216 are the same root cause in the statusline (RFE1 bytes misread as a SQLite page count, producing a ~2.47B/3.3B pattern count). Native drivers opening an RFE1 store without a sniff is a recurring pattern rather than a one-off.

Suggested fix direction
  • Option A — Update @claude-flow/memory to reuse the existing readFileMaybeEncrypted / isEncryptedBlob primitives from fs-secure.js so it can open RFE1 stores directly.
  • Option B — On first start under 3.33.0, detect a populated encrypted memory.db beside an empty agentdb-memory.db and run a one-time decrypt-and-import migration.
  • Option C — At minimum, emit a prominent warning when an encrypted memory.db sits beside an empty agentdb-memory.db, and suppress the memory init hint whenever the store is encrypted.

Recommendation: Option B as the real fix, with Option C shipped immediately as a low-risk guard against destructive user action.

Workaround for other users

The corpus can be recovered manually: decrypt .swarm/memory.db via fs-secure.js and import rows from memory_entries into agentdb-memory.db.

Two gotchas found doing this:

  • agentdb-memory.db has an extra provenance_type column; legacy rows need 'unknown'.
  • Make the import idempotent on id (INSERT OR IGNORE) so a partial run can be retried safely.

We recovered 8,923 entries across 33 namespaces this way with 99.6% embedding coverage and zero failures, so the data is fully salvageable once you know the store split exists.

Note on reproducing against our environment

Our local instance has since been repaired using the workaround above, so agentdb-memory.db is now populated (~83 MB) rather than empty. The empty-store state described in steps 3–4 is what we observed before recovery. Happy to provide the pre-recovery artifacts or the migration script if useful.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with memory-bridge.js and graph-edge-writer.js, then read the existing RFE1 handling in fs-secure.js, memory-initializer.js, and doctor.js. Reproduce the MCP memory calls against an encrypted store and verify that existing records are read or migrated, encrypted stores trigger a clear warning, and no destructive memory init hint is shown.

Written by the indexing model from the issue text.

Assessment

Tech stack
nodejs, sqlite, typescript
Domain
databases, security
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.