RVF 0.3.4 does not persist INDEX_SEG; reopened 28k-vector stores take >60s on first query
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 4.5k
- Forks
- 603
- Avg merge
- 23h 32m
- Merged PRs (30d)
- 59
Description
Summary
The current public packages (@ruvector/rvf@0.3.4, @ruvector/rvf-node@0.2.3, both current latest on 2026-07-30) do not emit a persisted index / INDEX_SEG when an RVF store is created and populated through the documented RvfDatabase.create() + ingestBatch() path.
This makes reopened large stores pay an extreme first-query cost. In RuvNet Brain's real source corpus:
- 1,466 vectors / 4.4 MB: first query ~5.5s, second query 3-15ms
- 13,764 vectors / 43 MB: first query exceeded 60s under the measured run
- 28,152 vectors / 83 MB: first query exceeded 60s under the measured run
The same stores open in 9-105ms. The delay occurs on the first db.query().
Segment proof
db.segments() on the 28,152-vector store reports 330 segments containing only:
manifestvecwitnessjournal
There is no index segment. The 1,466-vector store likewise has 23 segments and no index.
A controlled 100-vector reproduction also emits no index:
const db = await RvfDatabase.create(file, {
dimensions: 4,
metric: 'cosine',
m: 16,
efConstruction: 200,
overwrite: true,
});
await db.ingestBatch(rows);
console.log(await db.segments());
await db.compact();
console.log(await db.segments());
Before compact: manifest, vec, witness, manifest.
After compact: vec, witness, witness.
Still no index.
Expected
The documented HNSW creation options (m, efConstruction) should produce a durable INDEX_SEG, and openReadonly() should use it so first-query latency is near steady-state query latency rather than scaling into tens of seconds.
If the index is intentionally embedded inside VEC_SEG, please document that layout and investigate why the first query scales this way after reopen. The public README currently identifies 0x02 INDEX_SEG as the HNSW graph segment.
Impact
This blocks an interactive <4s search SLO and caused shared MCP worker timeouts at 30-240 seconds. Raising the timeout is not an acceptable workaround.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the documented RvfDatabase.create(), ingestBatch(), openReadonly(), segments(), and compact() paths, then reproduce the 100-vector case and inspect the README's INDEX_SEG description. Done means the documented creation path produces a durable index used after reopen, or the VEC_SEG layout is documented and first-query latency no longer scales into the reported timeouts.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- node.js, rust
- Domain
- databases, performance
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100