facebook / facebook/zstd

Compression fails on linux/i386 with ZSTD_maxCLevel + ZSTD_c_nbWorkers

Abierto
#4,628 2 comentarios 1 reacción 0 asignados Ver en GitHub
Lenguaje dominante
C
Estrellas
27.9k
Forks
2.6k
Merge medio
1 d 3 h
PR fusionados (30 d)
8

Descripción

**Describe the bug**

Compression fails on 32-bit architectures at level 22 with threading.
I've traced the problem to a `malloc(1<<31)` failure.

**To Reproduce**

On linux/i386, such as `docker run -it --platform=linux/i386 debian:testing`, create the following source file:

example.c (click to expand/collapse)

```c
#include
#include

int main(void)
{
size_t zret;

unsigned char const buf_in[] = "Sample Input Buffer Content";
ZSTD_inBuffer zin = { buf_in, sizeof(buf_in), 0 };
unsigned char buf_out[4096];
ZSTD_outBuffer zout = { buf_out, sizeof(buf_out), 0 };

ZSTD_CStream* zcs = ZSTD_createCStream();

/* Activate ZSTD_MULTITHREAD code paths. */
ZSTD_CCtx_setParameter(zcs, ZSTD_c_nbWorkers, 1);

/* Set maximum compression level (22). */
zret = ZSTD_initCStream(zcs, ZSTD_maxCLevel());
assert(!ZSTD_isError(zret) && "ZSTD_initCStream failed");

/* Try compressing. */
zret = ZSTD_compressStream(zcs, &zout, &zin);
assert(!ZSTD_isError(zret) && "ZSTD_compressStream failed");

ZSTD_freeCStream(zcs);

return 0;
}
```

Compile and link against a distro-packaged zstd 1.5.7. Run the binary:

```console
$ gcc -g example.c -o example -lzstd && ./example
example: example.c:24: main: Assertion `!ZSTD_isError(zret) && "ZSTD_compressStream failed"' failed.
```

Compression fails.

**Expected behavior**

Compression succeeds.

**Additional context**

The failure occurs on [this line](https://github.com/facebook/zstd/blob/v1.5.7/lib/compress/zstdmt_compress.c#L1331) with `capacity == 0x80000000`, `sectionsSize == 0x20000000`, `mtctx->targetSectionSize == 0x20000000`, `nbWorkers == 1`, `nbSlackBuffers == 3`.

Guía de contribución

Abrir la guía de contribución

Evaluación

Este issue todavía no se ha evaluado.

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.