lance-format / lance-format/lance
Support nested column references in UpdateBuilder::set()
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 7.1k
- Forks
- 852
- Avg merge
- 3d 18h
- Merged PRs (30d)
- 272
Description
UpdateBuilder::set() rejects any column name containing ., so a nested field cannot be updated directly — it can only be changed by setting its whole top-level struct column.
Location — rust/lance/src/dataset/write/update.rs, in set():
// TODO: support nested column references. This is mostly blocked on the
// ability to insert them into the RecordBatch properly.
if column.as_ref().contains('.') {
return Err(Error::not_supported_source(
format!(
"Nested column references are not yet supported. Referenced: {}",
column.as_ref(),
)
.into(),
));
}
Impact
- To change
s.x, callers mustSET s = <whole struct>, which is awkward and rewrites the entire struct value. - This is what made #7412 necessary: a whole-struct update resolves to the parent field id, so a scalar index on a nested child field (
s.x) had to be handled specially to avoid being wrongly extended over the rewritten fragment.
Ask
Support set("s.x", <expr>) for nested leaf columns. The existing TODO notes this is mostly blocked on inserting nested columns into the RecordBatch properly. Once supported, the index-coverage logic can target the leaf field directly instead of conservatively expanding the whole struct subtree.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in rust/lance/src/dataset/write/update.rs at UpdateBuilder::set() and trace how update expressions are inserted into a RecordBatch. Study the existing handling of whole-struct updates and the index-coverage logic described in the issue. Done means set("s.x", ) updates the nested leaf directly without rejecting the reference or rewriting the whole struct.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- databases
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100