ruvnet / ruvnet/RuVector

RVF SDK silently drops metadata; native metadata is not durable or returned

Open
#704 4 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Rust
Stars
4.5k
Forks
603
Avg merge
23h 32m
Merged PRs (30d)
59

Description

Versions

  • @ruvector/rvf@0.2.3
  • @ruvector/rvf-node@0.2.0
  • darwin-arm64 native binary 0.1.7
  • Node 26.4.0, macOS arm64

Public SDK reproduction

const db = await RvfDatabase.create(path, {
  dimensions: 2,
  metric: 'cosine',
}, 'node');

await db.ingestBatch([
  { id: 'red-item', vector: new Float32Array([1, 0]), metadata: { color: 'red' } },
  { id: 'blue-item', vector: new Float32Array([0, 1]), metadata: { color: 'blue' } },
]);

Observed:

  • Ingest reports both vectors accepted.
  • Unfiltered results contain no metadata.
  • A field-0 equality filter for red returns no results.

Native N-API reproduction

Using the low-level binding, pass one positional metadata record per vector:

db.ingestBatch(
  new Float32Array([1, 0, 0, 1]),
  [101, 102],
  [
    { fieldId: 0, valueType: 'string', value: 'red' },
    { fieldId: 0, valueType: 'string', value: 'blue' },
  ],
);

A field-0 red filter works while that database instance remains open. However:

  • Results still contain only ID and distance.
  • After close/reopen, the same filter returns no results.

Root cause

  • npm/packages/rvf/src/backend.ts accepts RvfIngestEntry.metadata but calls native ingestBatch(flat, ids) without metadata.
  • Native SearchResult contains only ID/distance.
  • Runtime inserts metadata into an in-memory MetadataStore, but no metadata segment is written and reloaded.
  • The SDK filter declaration omits the native parser's required valueType.

Impact

Applications can successfully ingest records while silently losing metadata. Filtering may appear to work through the native API and then stop working after restart, which can produce incomplete or incorrect search results without an error.

Requested fix

As an immediate safety measure, reject non-empty SDK metadata until it is supported rather than silently accepting it.

A complete fix needs:

  • A per-vector metadata shape across the SDK and N-API.
  • Metadata segment persistence and reload.
  • Aligned filter types, including valueType.
  • Either metadata-bearing search results or getMetadata(id).
  • Close/reopen tests, including vectors with unequal metadata-field counts.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with npm/packages/rvf/src/backend.ts and reproduce the public SDK and native N-API cases described in the issue. Trace the native SearchResult, MetadataStore, metadata segment persistence, and filter parsing, then add close/reopen coverage including unequal metadata-field counts. Done means metadata is not silently dropped, filters remain effective after reopening, and callers can retrieve metadata or receive metadata-bearing results.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript, node.js, rust
Domain
api, backend, database
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.