lance-format / lance-format/lance

bug: Flatten panics on a multivector batch without a row-id column

Open Beginner friendly
#9,425 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug
Dominant language
Rust
Stars
7.1k
Forks
852
Avg merge
3d 18h
Merged PRs (30d)
272

Description

Description

Flatten::transform in rust/lance-index/src/vector/transform.rs flattens a List<FixedSizeList> column by repeating each row id once per inner vector, and it reads the row ids with the indexing operator:

DataType::List(_) => {
    let row_ids = batch[ROW_ID].as_primitive::<UInt64Type>();

RecordBatch's Index impl panics when the column is absent, so a multivector batch without _rowid takes the caller down instead of returning an error. Flatten is public with a public constructor, and the fixed-size-list branch immediately above returns the batch untouched without needing the column, so nothing in the type or the contract says the column has to be there.

The other transformers in this file report their missing or mistyped columns as errors, and rust/AGENTS.md asks for column_by_name() in production code with batch["column"] left to tests.

Expected behavior

Return an error naming the missing column, like the neighbouring transformers.

Lance version

13.0.0-beta.4 (main)

Language binding

Rust

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in rust/lance-index/src/vector/transform.rs at Flatten::transform, then inspect the neighboring transformers and rust/AGENTS.md for the expected column access pattern. Reproduce a List batch without _rowid and verify that the transform returns an error naming the missing column instead of panicking.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
data
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
84/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.