weaviate / weaviate/typescript-client

data.ingest() silently stores empty objects for the NonReferenceInputs form its type signature accepts

Open Beginner friendly
#456 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
TypeScript
Stars
101
Forks
38
Avg merge
4d 12h
Merged PRs (30d)
2

Description

Summary

collection.data.ingest() accepts NonReferenceInputs<T> (the unwrapped { title: 'x' } form) in its type signature, but silently stores objects with no properties when given that form. The call reports success: UUIDs are returned, errors is empty, and hasErrors is false.

collection.data.insertMany() handles the same unwrapped input correctly, so the two batch entry points disagree on an input shape they both claim to accept.

This is silent data loss. A caller following the type signature gets a clean success response and an empty collection.

The type signature accepts the unwrapped form

src/collections/data/index.ts:88:

ingest: (objs: Iterable<DataObject<T> | NonReferenceInputs<T>>) => Promise<BatchObjectsReturn<T>>;

The implementation does not normalize it

src/collections/data/index.ts:251-257:

for (const obj of objs) {
  // eslint-disable-next-line no-await-in-loop
  await batching.addObject({
    collection: name,
    ...obj,          // <-- spread directly
    tenant,
  });
}

For a DataObject<T> the spread yields { collection, properties, tenant } and works. For a NonReferenceInputs<T> it yields { collection, title, tenant }properties is never set, so the object is created empty.

Compare insert in the same file, :287, which normalizes exactly this case:

parseObject(
  obj ? (DataGuards.isDataObject(obj) ? obj : ({ properties: obj } as InsertObject<T>)) : obj
),

DataGuards.isDataObject is defined at src/collections/serialize/index.ts:355 and is already the established guard for this discrimination. ingest never calls it.

Reproduction

import weaviate from 'weaviate-client';

const client = await weaviate.connectToLocal();

async function fresh(name) {
  if (await client.collections.exists(name)) await client.collections.delete(name);
  return client.collections.create({
    name,
    properties: [{ name: 'title', dataType: 'text' }],
    vectorizers: weaviate.configure.vectors.selfProvided(),
  });
}

// A — unwrapped NonReferenceInputs form
const a = await fresh('ReproIngestUnwrapped');
const ra = await a.data.ingest([{ title: 'alpha' }, { title: 'beta' }]);
console.log('uuids:', Object.keys(ra.uuids).length, 'errors:', Object.keys(ra.errors).length, 'hasErrors:', ra.hasErrors);
for await (const o of a.iterator()) console.log('STORED:', JSON.stringify(o.properties));

// B — wrapped DataObject form
const b = await fresh('ReproIngestWrapped');
await b.data.ingest([{ properties: { title: 'alpha' } }, { properties: { title: 'beta' } }]);
for await (const o of b.iterator()) console.log('STORED:', JSON.stringify(o.properties));

// C — insertMany with the same unwrapped form
const c = await fresh('ReproInsertMany');
await c.data.insertMany([{ title: 'alpha' }, { title: 'beta' }]);
for await (const o of c.iterator()) console.log('STORED:', JSON.stringify(o.properties));
Actual output
A: data.ingest([{ title: "..." }])          <- NonReferenceInputs form
  reported uuids   : 2
  reported errors  : 0
  hasErrors        : false
  STORED PROPERTIES: {}
  STORED PROPERTIES: {}

B: data.ingest([{ properties: { title } }]) <- DataObject form
  reported uuids   : 2
  STORED PROPERTIES: {"title":"beta"}
  STORED PROPERTIES: {"title":"alpha"}

C: data.insertMany([{ title: "..." }])      <- same unwrapped form
  STORED PROPERTIES: {"title":"alpha"}
  STORED PROPERTIES: {"title":"beta"}
Expected

Case A should store {"title":"alpha"} and {"title":"beta"}, matching cases B and C.

If the unwrapped form is not intended to be supported by ingest, then the type signature at :88 should drop NonReferenceInputs<T> so this fails at compile time rather than silently at runtime.

Suggested fix

Apply the same normalization insert already uses, inside the ingest loop:

for (const obj of objs) {
  await batching.addObject({
    collection: name,
    ...(DataGuards.isDataObject(obj) ? obj : { properties: obj }),
    tenant,
  });
}

Environment

  • weaviate-client 3.13.1
  • Weaviate server 1.38.0 (cr.weaviate.io/semitechnologies/weaviate:1.38.0, anonymous access, self-provided vectors)
  • Node.js v22.14.0, macOS

Note

This surfaced while updating the Weaviate quickstart documentation to use data.ingest(). Every plain property list had to be rewritten as .map((properties) => ({ properties })) to work, which is what led back to the signature mismatch.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in src/collections/data/index.ts at the ingest signature around line 88 and the loop around lines 251-257. Compare its handling with insert around line 287 and the DataGuards.isDataObject definition at src/collections/serialize/index.ts:355. Verify the reproduction for wrapped and unwrapped inputs; done means unwrapped properties are stored and the reported success behavior remains correct.

Written by the indexing model from the issue text.

Assessment

Tech stack
typescript
Domain
database
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
82/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.