lance-format / lance-format/lance
[MemTable] Scan with sealed, flushed but not merged MemTables
Open
Nobody has claimed this yet.
enhancement
- Dominant language
- Rust
- Stars
- 7.1k
- Forks
- 852
- Avg merge
- 3d 18h
- Merged PRs (30d)
- 272
Description
When scanning, take into account of all the sealed, flushed but not merged MemTables (a.k.a. SSTables before the last layer). The scanner should add a related flag, and when enabled, the read will merge results of all such MemTables stored in the MemWAL index.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Locate the scanner and the MemWAL index, then trace how sealed, flushed, and unmerged MemTables are represented. Define the scanner flag's enabled behavior so reads merge all such MemTables, and verify that scans include those results without the flag changing existing behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- databases
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 39/100