RVF SDK silently drops metadata; native metadata is not durable or returned
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 4.5k
- Forks
- 603
- Avg merge
- 23h 32m
- Merged PRs (30d)
- 59
Description
Versions
@ruvector/rvf@0.2.3@ruvector/rvf-node@0.2.0- darwin-arm64 native binary
0.1.7 - Node 26.4.0, macOS arm64
Public SDK reproduction
const db = await RvfDatabase.create(path, {
dimensions: 2,
metric: 'cosine',
}, 'node');
await db.ingestBatch([
{ id: 'red-item', vector: new Float32Array([1, 0]), metadata: { color: 'red' } },
{ id: 'blue-item', vector: new Float32Array([0, 1]), metadata: { color: 'blue' } },
]);
Observed:
- Ingest reports both vectors accepted.
- Unfiltered results contain no metadata.
- A field-0 equality filter for
redreturns no results.
Native N-API reproduction
Using the low-level binding, pass one positional metadata record per vector:
db.ingestBatch(
new Float32Array([1, 0, 0, 1]),
[101, 102],
[
{ fieldId: 0, valueType: 'string', value: 'red' },
{ fieldId: 0, valueType: 'string', value: 'blue' },
],
);
A field-0 red filter works while that database instance remains open. However:
- Results still contain only ID and distance.
- After close/reopen, the same filter returns no results.
Root cause
npm/packages/rvf/src/backend.tsacceptsRvfIngestEntry.metadatabut calls nativeingestBatch(flat, ids)without metadata.- Native
SearchResultcontains only ID/distance. - Runtime inserts metadata into an in-memory
MetadataStore, but no metadata segment is written and reloaded. - The SDK filter declaration omits the native parser's required
valueType.
Impact
Applications can successfully ingest records while silently losing metadata. Filtering may appear to work through the native API and then stop working after restart, which can produce incomplete or incorrect search results without an error.
Requested fix
As an immediate safety measure, reject non-empty SDK metadata until it is supported rather than silently accepting it.
A complete fix needs:
- A per-vector metadata shape across the SDK and N-API.
- Metadata segment persistence and reload.
- Aligned filter types, including
valueType. - Either metadata-bearing search results or
getMetadata(id). - Close/reopen tests, including vectors with unequal metadata-field counts.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with npm/packages/rvf/src/backend.ts and reproduce the public SDK and native N-API cases described in the issue. Trace the native SearchResult, MetadataStore, metadata segment persistence, and filter parsing, then add close/reopen coverage including unequal metadata-field counts. Done means metadata is not silently dropped, filters remain effective after reopening, and callers can retrieve metadata or receive metadata-bearing results.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- javascript, node.js, rust
- Domain
- api, backend, database
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100