lance-format / lance-format/lance
Implement scalar-indexed joins on v2 merge_insert path
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 7.1k
- Forks
- 852
- Avg merge
- 3d 18h
- Merged PRs (30d)
- 272
Description
What to build
Implement the indexed join approach chosen in #6443 so that upserts with a
scalar index on the join key use the v2 path instead of falling back to v1.
Remove the scalar index fallback from can_use_create_plan. The use_index
parameter is retained as an escape hatch that controls whether the indexed
join node is used in the plan.
See milestone description for full PRD.
Acceptance criteria
- Upserts with scalar-indexed join keys use the v2 path (verified via
explain_plan) - Existing indexed merge insert tests pass unchanged
- New
explain_plantest confirms indexed join node appears in plan -
use_index=falsestill works as escape hatch (bypasses index, uses full scan join) - Conflict detection (Bloom filter) works for indexed operations
Blocked by
- #6443 (research spike must complete first)
User stories addressed
- 3: indexed upserts use v2 path
- 4: one fewer case on v1, moving toward single code path
- 5:
use_indexretained as escape hatch
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reading the research spike in #6443 and the merge_insert planning path around can_use_create_plan. Trace explain_plan for scalar-indexed join keys, then verify the indexed join node, the use_index=false full-scan escape hatch, unchanged indexed merge insert tests, and Bloom-filter conflict detection. Done means all listed acceptance criteria pass.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- databases
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100