Feature request: native multithreaded decompression for standard frames
- Lenguaje dominante
- C
- Estrellas
- 27.9k
- Forks
- 2.6k
- Merge medio
- 1 d 3 h
- PR fusionados (30 d)
- 8
Descripción
**Is your feature request related to a problem? Please describe.**
Some workloads compress a large archive once and decompress it many times. Zstandard supports multithreaded compression through `-T#`, but decompression remains single-threaded and can become a bottleneck on modern multicore systems with fast storage.
In one test with an approximately 4 GB archive, multithreaded zstd compression took about 21 seconds, while single-threaded zstd decompression plus tar extraction took about 26 seconds, compared with about 31 seconds using pigz. Compression completing faster than the extraction path is a useful signal that decompression is worth optimizing, although the extraction measurement also includes tar and filesystem writes.
This is a follow-up to #2470, where several users requested that multithreaded decompression be reconsidered after the issue was closed. Related discussions also exist in #2245 and #2499.
**Describe the solution you would like**
Add opt-in multithreaded decompression to the standard zstd CLI and API, ideally for streams produced by the normal zstd encoder. A CLI interface such as `zstd -d -T0` would be intuitive and consistent with compression.
**Describe alternatives you have considered**
`pzstd`, multiple independent frames, or manually splitting the input can enable parallel decompression, but these approaches require a different encoding strategy and may affect compression ratio or integration complexity.
Guía de contribución
Evaluación
Este issue todavía no se ha evaluado.