lance-format / lance-format/lance

Coalesce IO requests in an IO scheduler

Open
#1,505 3 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

performance rust
Dominant language
Rust
Stars
7.1k
Forks
852
Avg merge
3d 18h
Merged PRs (30d)
272

Description

Right now, we coalesce requests if they are within some size, but this only happens if we have context for them. Sometimes we might have two contiguous requests that could be combined, but they don't have context to know about each other. For example, if three i32 columns are being read in parallel (each request is 1024 * 4 = 4096), their buffers might be right next to each other, in which case the read could be just one 12Kb read.

This kind of scheduling is important to be competitive against row-based formats for random reads, since right now our row-level reads are O(num columns retrieved). To make this more useful, we might consider rearranging buffers so small column types are placed together.

We might take some inspiration from the design of Linux IO schedulers (such as Kyber). See: https://lwn.net/Articles/720675/ (The existence of these likely means that we just have to implement this for object storage, not local filesystems. Although it might be interesting to come up with recommendations on the best Linux IO scheduler settings for Lance.)

In addition, object stores often recommend cancelling and retrying a request if it is taking unusually long. This can help reduce p99 latencies. There is some research into this as well: https://www.sciencedirect.com/science/article/abs/pii/S0167819118302138. We should see if it's worth integrating this into the scheduler, if it's not already handled by object store.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

No file or test is named. Start by locating the IO scheduler and its request-coalescing path, then review how local filesystem and object-store requests are handled. Done would require a defined scheduling design, measurable coalescing or latency improvements, and tests or benchmarks covering the proposed behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, linux, rust
Domain
infrastructure, performance
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.