ruvnet / ruvnet/ruflo

Node 24→26 with reused npx cache: AgentDB swallows better-sqlite3 ABI mismatch and prints green sql.js success

Open
#3,175 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
TypeScript
Stars
72.8k
Forks
8.6k
Avg merge
2d 23h
Merged PRs (30d)
82

Description

## Summary

On current `ruflo@3.38.21`, changing the Node runtime from 24 to 26 while reusing an existing `npx` cache leaves the cached `better-sqlite3` native addon compiled for Node 24's ABI. AgentDB correctly detects that the native probe cannot open, but it then discards the actual exception and prints a green success line for the `sql.js` fallback:

```text
✅ Using sql.js (WASM SQLite, no build tools required)
```

There is no warning that the preferred native driver failed, no ABI/path information, and no indication that persistence semantics have changed.

This is a current-path regression/incomplete coverage of #901 and a distinct entry condition for the unresolved native-driver work in #2968. It is not #2219's old dependency-version problem: the installed `better-sqlite3@12.11.1` supports Node 26, but the binary in the reused npx cache was built under Node 24.

## Environment

```text
macOS arm64
Node v26.8.1
process.versions.modules = 147
ruflo 3.38.21 (current npm latest)
agentdb 3.0.0-alpha.20
better-sqlite3 12.11.1
launcher: npx -y ruflo@latest mcp start
cache: $HOME/.npm/_npx/
```

The same npx cache had previously been populated and used under Node v24.14.1 (ABI 137).

## Exact evidence

Loading the cached native addon under the current Node process fails:

```text
The module '$HOME/.npm/_npx//node_modules/better-sqlite3/build/Release/better_sqlite3.node'
was compiled against a different Node.js version using
NODE_MODULE_VERSION 137. This version of Node.js requires
NODE_MODULE_VERSION 147.
```

Calling AgentDB's driver selector against that same package:

```bash
node --input-type=module -e \
"const m=await import('./node_modules/agentdb/dist/src/db-fallback.js'); await m.getDatabaseImplementation()"
```

prints only:

```text
✅ Using sql.js (WASM SQLite, no build tools required)
```

and exits 0.

## Current source cause

Current AgentDB main still has a bare catch in
[`src/db-fallback.ts`](https://github.com/ruvnet/agentdb/blob/main/src/db-fallback.ts):

```ts
try {
const mod: any = await import('better-sqlite3');
// ...
const probe = new BetterSqlite3(':memory:');
probe.close();
// ...
} catch {
// Not installed or failed to load → fall through to sql.js.
}
```

It then labels fallback selection with a green check:

```ts
console.error('✅ Using sql.js (WASM SQLite, no build tools required)');
```

[`src/core/AgentDB.ts`](https://github.com/ruvnet/agentdb/blob/main/src/core/AgentDB.ts) similarly catches the native error and replaces it with the generic message `better-sqlite3 not available`.

The probe itself is good: it catches the important case where importing the JS wrapper succeeds but opening the native binding fails. The defect is throwing away the reason and representing an unrequested degradation as success.

## Why the existing issues do not close this

- #901 is closed and says an ABI mismatch should produce clear recovery guidance. The current AgentDB v3 path does not do that.
- #2968 remains open. Its reported entry condition is a skipped postinstall/missing binary. The CLI false-success and doctor signal were partially addressed, but the maintainer explicitly left install/native-driver policy open. A stale npx binary after a Node-major switch is another current entry condition.
- #2735 added demotion logging to Ruflo's memory bridge, but this separate AgentDB driver-selector catch still discards the underlying exception.
- #2219 fixed an incompatible dependency floor. Here the package version is current and compatible; only its cached native artifact belongs to the previous Node ABI.

## Impact

- A persistent Ruflo MCP server can start on a materially different database implementation without making the operator aware.
- Native WAL capability and performance disappear.
- When a managed database has WAL/SHM sidecars, later operations may correctly refuse the unsafe sql.js path, but the initiating cause is hidden and users see only the downstream refusal.
- Other call paths can mistake a degraded fallback for a healthy native installation.
- Multiple projects using `npx ruflo@latest` can reuse the same stale cache, widening the failure across every MCP process after a Node switch.

This report does **not** claim the database was corrupted. In the observed case the downstream WAL safety guard refused the operation and protected it.

## Proposed implementation

### 1. Preserve and classify the native load error

Replace the bare `catch {}` with `catch (error)`. Record a structured selection result:

```ts
type DatabaseSelection = {
implementation: 'better-sqlite3' | 'sql.js';
degraded: boolean;
intentional: boolean;
nativeError?: {
code?: string;
message: string;
nodeVersion: string;
runtimeAbi: string;
addonPath?: string;
compiledAbi?: string;
};
};
```

Recognize at least:

- `NODE_MODULE_VERSION` / ABI mismatch;
- native binding missing;
- architecture mismatch;
- install/postinstall failure;
- database-open or migration errors, which must not be mislabeled as “module unavailable”.

### 2. Never display unrequested fallback as green success

For automatic fallback, emit a once-per-process warning containing:

- native driver name and version;
- current Node version/ABI;
- compiled ABI when parsable;
- resolved addon/cache path;
- selected fallback;
- durability/capability consequences;
- exact supported remediation.

Reserve the green line for either native success or an explicitly requested `AGENTDB_FORCE_SQLJS=1` selection.

### 3. Fail closed for persistent managed stores unless fallback is explicit

For a file-backed managed AgentDB, an ABI/load failure should not silently change the storage contract. Either:

- repair/reinstall the native package before opening the store; or
- return a typed nonzero startup failure with remediation.

Allow sql.js automatically only for environments where it is the declared provider, or behind explicit `AGENTDB_FORCE_SQLJS=1`. Existing WAL/journal guards remain mandatory.

### 4. Make doctor inspect the exact active runtime

Doctor should execute the same real `:memory:` native probe through the package instance that the MCP process will load. Its structured result should include the original failure cause and active provider. An ABI mismatch should make native persistence unhealthy, not merely be inferred later from schema/table counts.

### 5. Add an npx ABI-transition preflight

Add an integration test and startup preflight for this lifecycle:

1. populate/reuse an npx package cache under Node 24;
2. start the same package/cache under Node 26;
3. detect that the native addon belongs to ABI 137 while the runtime needs ABI 147;
4. either perform a supported package-local rebuild/refetch and re-probe, or stop with a typed nonzero error before opening a persistent DB.

The cache or verified-native marker should be keyed by at least package version + platform + architecture + Node module ABI. A “latest” package version alone is insufficient because native bytes can become stale without the JS package version changing.

## Acceptance tests

1. Mock the native probe throwing the exact `NODE_MODULE_VERSION 137 ... requires 147` error; assert the original cause is preserved and emitted.
2. Assert an automatic fallback never prints a green success check.
3. Assert explicit `AGENTDB_FORCE_SQLJS=1` remains supported and is reported as intentional.
4. Assert a file-backed managed store refuses automatic fallback with a typed nonzero error.
5. Run a Node 24→26 reused-cache integration case and prove either repair + native re-probe or deterministic refusal.
6. Assert doctor uses the active package's actual native probe and reports the runtime ABI, addon ABI/path, and selected provider.
7. Assert a WAL/SHM-bearing store is never opened or rewritten by sql.js.

## Related

- #2968
- #901
- #2735
- #2219
- #360

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with src/db-fallback.ts and src/core/AgentDB.ts, then reproduce the provided driver-selector command with the cached Node 24/26 setup. Trace how the native probe error is handled and how the sql.js message is emitted. Done means the original ABI or load cause is reported, automatic persistent fallback fails safely, explicit AGENTDB_FORCE_SQLJS=1 remains intentional, and the listed acceptance tests cover doctor and the cache transition.

Written by the indexing model from the issue text.

Assessment

Tech stack
node.js, sqlite, typescript
Domain
backend, databases
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.