facebook / facebook/zstd

Multithreaded improvements

Abierto
#2,499 2 comentarios 2 reacciones 0 asignados Ver en GitHub
question
Lenguaje dominante
C
Estrellas
27.9k
Forks
2.6k
Merge medio
1 d 3 h
PR fusionados (30 d)
8

Descripción

It looks like the the multithreaded implementation in the base zstd is a bit slower than the other available threading implementations:
zstd-mt : https://github.com/mcmilk/zstdmt/releases
pzstd: https://github.com/facebook/zstd/tree/dev/contrib/pzstd

One issue is the cpu detection, with -T0 zstd doesn't count hyperthreading cores, but at higher compression ratios they can provide additional speedup.

Another issue is decompression doesn't use threads, even adding a couple threads dedicated to reading and writing would allow speedup of single block streams. Perhaps decompression is fast enough that true multithreaded decompression per block (like plzip, lbzip2, or pbzip2) would not provide much benefit.

Jon

Guía de contribución

Abrir la guía de contribución

Evaluación

Este issue todavía no se ha evaluado.

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.