NVIDIA / NVIDIA/NemoClaw

`openclaw memory index --force` fails with "unable to open database file" (FTS5 + ATTACH + transaction under node:sqlite)

Open
#9,015 5 comments 0 reactions 0 assignees View on GitHub
integration: openclaw
Dominant language
TypeScript
Stars
22.5k
Forks
3.1k
Avg merge
1d 1h
Merged PRs (30d)
715

Description

### Investigation Summary

- `openclaw memory index --force` fails every time with `Memory index failed (main): unable to
open database file`, reproduced on a fresh/empty `openclaw-agent.sqlite` as well, so it's not
pre-existing corruption.
- Instrumented `node:sqlite`'s `DatabaseSync.prototype.exec`/`prepare` to log on throw; the ex
act failing statement is `DROP TABLE IF EXISTS main.memory_index_chunks_fts`, called from `rep
laceVirtualTable()` inside `publishMemoryDatabaseTables()` in the bundled `manager-C0zXukeE.js
` (memory-core).
- Isolated a minimal, OpenClaw-independent repro: dropping an FTS5 virtual table fails with `S
QLITE_CANTOPEN` under Node's built-in `node:sqlite` specifically when another database is `ATT
ACH`ed *and* a `BEGIN IMMEDIATE` transaction is open — either condition alone does not reprodu
ce it.
- Ruled out network/config as the cause first: the configured Ollama embedding provider (`agen
ts.defaults.memorySearch` → `provider: "ollama"`, `remote.baseUrl`) is confirmed reachable and
correct via `openclaw memory status --deep` and direct `curl`/`node` probes against the same
endpoint.
- Unknown: whether this reproduces on non-experimental/newer Node `node:sqlite` builds, or onl
y on v22.23.1's implementation; have not tried a `better-sqlite3`-backed build for comparison.

### Description

**What happened:** `openclaw memory index --force` always fails with `Memory index failed (mai
n): unable to open database file`, so the memory search index can never be (re)built and `memory_search`/auto-recall stay unavailable ("Vector search: paused until memory is rebuilt").

**What I expected:** The full reindex to complete successfully and populate the FTS5/vector index.

### Reproduction Steps

1. Configure a working embedding provider for memory search, e.g.:
```json5
{
agents: {
defaults: {
memorySearch: {
provider: "ollama",
model: "qwen3-embedding:4b",
remote: { baseUrl: "http://:11434" },
},
},
},
}
```
2. Run:
```bash
openclaw memory index --force
```
3. Observe:
```
Memory index failed (main): unable to open database file
```
Minimal, OpenClaw-independent repro of the underlying `node:sqlite` behavior:

```js
const { DatabaseSync } = require('node:sqlite');

const db = new DatabaseSync('main.sqlite');
db.exec('CREATE VIRTUAL TABLE IF NOT EXISTS memory_index_chunks_fts USING fts5(text)');

const tempDb = new DatabaseSync('temp.sqlite');
tempDb.exec('CREATE TABLE dummy(x)');
tempDb.close();

db.prepare('ATTACH DATABASE ? AS memory_reindex').run('temp.sqlite');
db.exec('BEGIN IMMEDIATE');
db.exec('DROP TABLE IF EXISTS main.memory_index_chunks_fts');
// Throws: Error: unable to open database file (code: ERR_SQLITE_ERROR)
```

- `ATTACH` alone (no open transaction): does not reproduce.
- Open `BEGIN IMMEDIATE` transaction alone (no attached database): does not reproduce.
- Both together: reproduces consistently.
- Same repro with a plain (non-virtual) table instead of the FTS5 table: does not reproduce — appears specific to FTS5's shadow-table handling under `DatabaseSync`.

### Environment

- OS: Ubuntu 24.04.4 LTS (kernel `6.17.0-1014-nvidia`), aarch64
- Hardware: DGX Spark
- Node.js (inside sandbox): v22.23.1 — `node:sqlite` (`DatabaseSync`) is experimental in this build
- Docker: OpenShell driver `docker`, openshell version `0.0.85`
- NemoClaw: v0.0.103 (`nemoclaw --version` / sourceRevision `db31c286129e878c3356eed49f76ab259561e47e`)
- OpenClaw: 2026.7.1 (`2d2ddc4`)
- sqlite-vec: sqlite-vec-linux-arm64 0.1.9
- Sandbox: `sho-assistant`, GPU auto, policies: npm, pypi, huggingface, brew, brave, github, gmail, local-inference, openclaw-diagnostics-otel-local, openclaw-pricing, weather, discord

### Debug Output

`nemoclaw debug --quick --sandbox sho-assistant` reports all onboarding/setup steps `complete` with no errors (`preflight`, `gateway`, `sandbox`, `provider_selection`, `inference`, `openclaw`, `policies`) — sandbox and gateway health are not implicated; this is specific to the memory-index code path. Full tarball from `nemoclaw debug --output` is attached separately (auto-redacted; please do your own review of the tarball contents before attaching, since I generated it but haven't line-by-line audited it myself).

[nemoclaw-debug.tar.gz](https://github.com/user-attachments/files/31039757/nemoclaw-debug.tar.gz)

### Logs

```shell
Memory index failed (main): unable to open database file

Traced with `NODE_OPTIONS="--require ./trace-sqlite.cjs" openclaw memory index --force`, where the preload wraps `DatabaseSync.prototype.exec`/`prepare` to log on throw:


[TRACE exec FAIL] "DROP TABLE IF EXISTS main.memory_index_chunks_fts" | unable to open database file | ERR_SQLITE_ERROR
Memory index failed (main): unable to open database file

Relevant source location (bundled): `extensions/memory-core/src/memory/manager-db.ts`, function `publishMemoryDatabaseTables()` → `replaceVirtualTable()` — shipped as `dist/manager-C0zXukeE.js` in the OpenClaw runtime package.
```

### Checklist

- [x] I confirmed this bug is reproducible
- [x] I searched existing issues and this is not a duplicate

Contributor guide

Open the contributing guide

Research direction

Start in extensions/memory-core/src/memory/manager-db.ts at publishMemoryDatabaseTables() and replaceVirtualTable(), then run the minimal node:sqlite reproduction with ATTACH, BEGIN IMMEDIATE, and the FTS5 table. Compare behavior across supported Node builds or database backends as appropriate. Done means openclaw memory index --force completes and memory_search is available with the index populated.

Written by the indexing model from the issue text.

Assessment

Tech stack
node.js, sqlite, typescript
Domain
databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.