microsoft / microsoft/DiskANN

Reduce disk search latency by overlapping I/O with compute (PipeANN-style pipelining)

Open
#1,285 0 comments 0 reactions 1 assignee View on GitHub

@wuw92 is already working on this.

Since Jul 29, 2026.

enhancement rust
Dominant language
Rust
Stars
1.9k
Forks
454
Avg merge
3d 22h
Merged PRs (30d)
35

Description

Disk search is lockstep: each beam iteration issues its reads, waits for all of them, then computes. Per-query latency is therefore near the sum of I/O and compute rather than the max.

A node can only be scored after its read completes, so the two can't be reordered within one iteration — but they can be interleaved across iterations: issue this iteration's reads, then score the nodes that arrived from a previous one, so reads are in flight while the CPU works. This is what PipeANN does ("Achieving Low-Latency Graph-Based Vector Search via Aligning Best-First Search Algorithm with SSD", OSDI '25).

#769 attempted this and was closed for pushing pipelining complexity into the core search algorithm. The direction suggested there, which this issue tracks, is to keep it in the accessor instead (comment):

we don't necessarily need ExpandBeam to exhaust the iterator on each call. It can store some state in the accessor and return from the future when there is no more work to do. This will cause the outer loop to be entered, queue a fresh batch of nodes for the next beam - and the next call to ExpandBeam can continue processing results from a previous call.

The same comment notes two constraints for any implementation: back-pressure isn't expressible this way without an additional hook letting ExpandBeam choose how many nodes to pick, and a blocking wait_for_io "goes against the async philosophy style of the Rust code" — the returned future should suspend instead.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.