lance-format / lance-format/lance
feat: support conditional delete in merge-insert when_matched clause
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 7.1k
- Forks
- 852
- Avg merge
- 3d 18h
- Merged PRs (30d)
- 272
Description
Problem
WhenMatched supports conditional update through UpdateIf and UpdateIfExpr, but delete is only unconditional. Conditional delete exists solely on WhenNotMatchedBySource::DeleteIf, and the Python binding when_matched_delete() takes no condition.
This makes it impossible to express "delete the matched target row only when a condition holds" in a single merge-insert. Examples are tombstone-driven deletes (source.deleted = true) and stale-delete guards (source.last_update > target.last_update). The current workaround is a separate scan and delete pass, which is not atomic with the merge.
Expected behavior
Rust:
let job = dataset.merge_insert(&["id"])
.when_matched(WhenMatched::delete_if(&dataset, "source.deleted = true")?)
...
Python, mirroring when_matched_update_all(condition):
dataset.merge_insert("id") \
.when_matched_delete("source.deleted = true") \
.when_not_matched_insert_all() \
.execute(new_data)
The condition supports source. and target. qualified references like UpdateIf does. Matched rows failing the condition are left untouched. Bare when_matched_delete() keeps its current unconditional behavior.
Proposed implementation
- Add
WhenMatched::DeleteIf(String)andDeleteIfExpr(Expr)variants mirroringUpdateIfandUpdateIfExpr, with parsing deferred because the two execution paths resolvesource.andtarget.references against different schemas (relation-qualified columns on the standard plan, combined-schema struct fields on the indexed path). - Fast path: new CASE arms in
assign_action.rsemittingAction::Deletewhen a row is matched and the condition is true, falling through toAction::Nothingotherwise. Delete-only plans keep usingDeleteOnlyMergeInsertExecunchanged since the condition is baked into the action assignment. - Slow (indexed) path: reuse the
Merger's existingmatch_filter_exprcompilation and apply the mask to matched rows before recording deletions. - Python: optional
conditionparameter on the existingwhen_matched_delete, fully backward compatible.
Suggested coverage
- Rust tests for both execution paths (the fast-path plan and the indexed Merger path), asserting
num_deleted_rowscounts only condition-true matches and surviving rows are unmodified. Error case for non-boolean conditions. - Python test extending
test_merge_insert_when_matched_deletewith conditional cases.
Follow-up (out of scope here): Java MergeInsertParams.withMatchedDeleteIf(String) mirroring withNotMatchedBySourceDeleteIf.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the WhenMatched variants, assign_action.rs, and the existing Merger match_filter_expr path; compare the fast-path plan with the indexed path. Extend the Rust coverage for both paths and the non-boolean error, then extend test_merge_insert_when_matched_delete for Python. Done means conditional matches alone are deleted, other matched rows remain unchanged, and unconditional deletion still works.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, rust
- Domain
- databases
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Clearly specified
- Newbie friendliness
- 56/100