lance-format / lance-format/lance
Push limit into scalar index query
Open
Nobody has claimed this yet.
A-index
enhancement
- Dominant language
- Rust
- Stars
- 7.1k
- Forks
- 852
- Avg merge
- 3d 18h
- Merged PRs (30d)
- 272
Description
Sometimes when many rows can be returned a scalar index query might take some time. For example, a large range query against a btree index that hits most or all of the data. If there is a limit on the scan then we could potentially push this into the index itself so that we know we can short-circuit early.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by tracing the scalar index query path and how scan limits are currently handled. Determine where a limit could be passed into the index so range scans can stop early, then validate that limited queries return the expected rows without scanning unnecessary data.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- database
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100