facebook / facebook/zstd

Hope to Add a Read Buffer Option

Open
#4,453 0 comments 0 reactions 0 assignees View on GitHub
feature request
Dominant language
C
Stars
27.9k
Forks
2.6k
Avg merge
1d 3h
Merged PRs (30d)
8

Description

**Is your feature request related to a problem? Please describe.**

I have the following scenario:

A 12TB mechanical hard drive with a 1TB file compressed using zstd. During decompression, I noticed that the disk is both reading and writing at the same time — approximately 80MB/s read and 50MB/s write. However, if reading or writing separately, the speed can exceed 250MB/s. The CPU usage of zstd is around 20%.

I suspect the performance issue is due to disk seek operations. I would like to be able to sequentially read 1–5GB into memory first, then decompress to disk. I believe this would be faster. Thank you.

**Describe the solution you'd like**
Hope to Add a Read Buffer Option

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.