`openclaw memory index --force` fails with "unable to open database file" (FTS5 + ATTACH + transaction under node:sqlite)
- Dominant language
- TypeScript
- Stars
- 22.5k
- Forks
- 3.1k
- Avg merge
- 1d 1h
- Merged PRs (30d)
- 715
Description
### Investigation Summary
- `openclaw memory index --force` fails every time with `Memory index failed (main): unable to
open database file`, reproduced on a fresh/empty `openclaw-agent.sqlite` as well, so it's not
pre-existing corruption.
- Instrumented `node:sqlite`'s `DatabaseSync.prototype.exec`/`prepare` to log on throw; the ex
act failing statement is `DROP TABLE IF EXISTS main.memory_index_chunks_fts`, called from `rep
laceVirtualTable()` inside `publishMemoryDatabaseTables()` in the bundled `manager-C0zXukeE.js
` (memory-core).
- Isolated a minimal, OpenClaw-independent repro: dropping an FTS5 virtual table fails with `S
QLITE_CANTOPEN` under Node's built-in `node:sqlite` specifically when another database is `ATT
ACH`ed *and* a `BEGIN IMMEDIATE` transaction is open — either condition alone does not reprodu
ce it.
- Ruled out network/config as the cause first: the configured Ollama embedding provider (`agen
ts.defaults.memorySearch` → `provider: "ollama"`, `remote.baseUrl`) is confirmed reachable and
correct via `openclaw memory status --deep` and direct `curl`/`node` probes against the same
endpoint.
- Unknown: whether this reproduces on non-experimental/newer Node `node:sqlite` builds, or onl
y on v22.23.1's implementation; have not tried a `better-sqlite3`-backed build for comparison.
### Description
**What happened:** `openclaw memory index --force` always fails with `Memory index failed (mai
n): unable to open database file`, so the memory search index can never be (re)built and `memory_search`/auto-recall stay unavailable ("Vector search: paused until memory is rebuilt").
**What I expected:** The full reindex to complete successfully and populate the FTS5/vector index.
### Reproduction Steps
1. Configure a working embedding provider for memory search, e.g.:
```json5
{
agents: {
defaults: {
memorySearch: {
provider: "ollama",
model: "qwen3-embedding:4b",
remote: { baseUrl: "http://:11434" },
},
},
},
}
```
2. Run:
```bash
openclaw memory index --force
```
3. Observe:
```
Memory index failed (main): unable to open database file
```
Minimal, OpenClaw-independent repro of the underlying `node:sqlite` behavior:
```js
const { DatabaseSync } = require('node:sqlite');
const db = new DatabaseSync('main.sqlite');
db.exec('CREATE VIRTUAL TABLE IF NOT EXISTS memory_index_chunks_fts USING fts5(text)');
const tempDb = new DatabaseSync('temp.sqlite');
tempDb.exec('CREATE TABLE dummy(x)');
tempDb.close();
db.prepare('ATTACH DATABASE ? AS memory_reindex').run('temp.sqlite');
db.exec('BEGIN IMMEDIATE');
db.exec('DROP TABLE IF EXISTS main.memory_index_chunks_fts');
// Throws: Error: unable to open database file (code: ERR_SQLITE_ERROR)
```
- `ATTACH` alone (no open transaction): does not reproduce.
- Open `BEGIN IMMEDIATE` transaction alone (no attached database): does not reproduce.
- Both together: reproduces consistently.
- Same repro with a plain (non-virtual) table instead of the FTS5 table: does not reproduce — appears specific to FTS5's shadow-table handling under `DatabaseSync`.
### Environment
- OS: Ubuntu 24.04.4 LTS (kernel `6.17.0-1014-nvidia`), aarch64
- Hardware: DGX Spark
- Node.js (inside sandbox): v22.23.1 — `node:sqlite` (`DatabaseSync`) is experimental in this build
- Docker: OpenShell driver `docker`, openshell version `0.0.85`
- NemoClaw: v0.0.103 (`nemoclaw --version` / sourceRevision `db31c286129e878c3356eed49f76ab259561e47e`)
- OpenClaw: 2026.7.1 (`2d2ddc4`)
- sqlite-vec: sqlite-vec-linux-arm64 0.1.9
- Sandbox: `sho-assistant`, GPU auto, policies: npm, pypi, huggingface, brew, brave, github, gmail, local-inference, openclaw-diagnostics-otel-local, openclaw-pricing, weather, discord
### Debug Output
`nemoclaw debug --quick --sandbox sho-assistant` reports all onboarding/setup steps `complete` with no errors (`preflight`, `gateway`, `sandbox`, `provider_selection`, `inference`, `openclaw`, `policies`) — sandbox and gateway health are not implicated; this is specific to the memory-index code path. Full tarball from `nemoclaw debug --output` is attached separately (auto-redacted; please do your own review of the tarball contents before attaching, since I generated it but haven't line-by-line audited it myself).
[nemoclaw-debug.tar.gz](https://github.com/user-attachments/files/31039757/nemoclaw-debug.tar.gz)
### Logs
```shell
Memory index failed (main): unable to open database file
Traced with `NODE_OPTIONS="--require ./trace-sqlite.cjs" openclaw memory index --force`, where the preload wraps `DatabaseSync.prototype.exec`/`prepare` to log on throw:
[TRACE exec FAIL] "DROP TABLE IF EXISTS main.memory_index_chunks_fts" | unable to open database file | ERR_SQLITE_ERROR
Memory index failed (main): unable to open database file
Relevant source location (bundled): `extensions/memory-core/src/memory/manager-db.ts`, function `publishMemoryDatabaseTables()` → `replaceVirtualTable()` — shipped as `dist/manager-C0zXukeE.js` in the OpenClaw runtime package.
```
### Checklist
- [x] I confirmed this bug is reproducible
- [x] I searched existing issues and this is not a duplicate
Contributor guide
Research direction
Start in extensions/memory-core/src/memory/manager-db.ts at publishMemoryDatabaseTables() and replaceVirtualTable(), then run the minimal node:sqlite reproduction with ATTACH, BEGIN IMMEDIATE, and the FTS5 table. Compare behavior across supported Node builds or database backends as appropriate. Done means openclaw memory index --force completes and memory_search is available with the index populated.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- node.js, sqlite, typescript
- Domain
- databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100