microsoft / microsoft/garnet

2 GiB hash-index checkpoint I/O silently completes short at MAX_RW_COUNT on Linux/libaio

Open
#2,134 0 comments 0 reactions 1 assignee View on GitHub

@badrishc is already working on this.

Since Sep 18, 2026.

Dominant language
C#
Stars
12k
Forks
703
Avg merge
2d 19h
Merged PRs (30d)
36

Description

### Describe the bug

The main hash-index checkpoint and recovery paths can issue individual 2 GiB I/O requests through `NativeStorageDevice` on Linux. Linux caps these requests at `MAX_RW_COUNT`, but positive short completions are not checked against the requested size by the index checkpoint/recovery callbacks.

On a system with 4 KiB pages, both operations complete only **2,147,479,552 bytes** of a **2,147,483,648-byte** request. The final **4,096 bytes** are not written/read, yet the managed index checkpoint/recovery completion reports success.

### Steps to reproduce the bug

1. Start Garnet on Linux with 4 KiB pages, using libaio, a 2 GiB hash index (with start arg `--index 2g --index-max-size 2g`), and the read cache disabled.
2. Insert 4 million keys using `SET` (`shortio:00000000` through `shortio:03999999`), then run `SAVE`.
3. `SAVE` returns OK, but the saved index is incomplete: its main hash table’s last 4 KiB is zero-filled, omitting entries for existing keys such as shortio:00553110, which is still readable from the running server.

### Expected behavior

_No response_

### Screenshots

_No response_

### Release version

v2.1.7

### IDE

_No response_

### OS version

Linux

### Additional context

_No response_

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.