facebook / facebook/zstd

Compression fails on linux/i386 with ZSTD_maxCLevel + ZSTD_c_nbWorkers

Aperta
#4,628 2 commenti 1 reazione 0 assegnatari Vedi su GitHub
Lingua principale
C
Stelle
27.9k
Fork
2.6k
Merge medio
1g 3h
PR unite (30g)
8

Descrizione

**Describe the bug**

Compression fails on 32-bit architectures at level 22 with threading.
I've traced the problem to a `malloc(1<<31)` failure.

**To Reproduce**

On linux/i386, such as `docker run -it --platform=linux/i386 debian:testing`, create the following source file:

example.c (click to expand/collapse)

```c
#include
#include

int main(void)
{
size_t zret;

unsigned char const buf_in[] = "Sample Input Buffer Content";
ZSTD_inBuffer zin = { buf_in, sizeof(buf_in), 0 };
unsigned char buf_out[4096];
ZSTD_outBuffer zout = { buf_out, sizeof(buf_out), 0 };

ZSTD_CStream* zcs = ZSTD_createCStream();

/* Activate ZSTD_MULTITHREAD code paths. */
ZSTD_CCtx_setParameter(zcs, ZSTD_c_nbWorkers, 1);

/* Set maximum compression level (22). */
zret = ZSTD_initCStream(zcs, ZSTD_maxCLevel());
assert(!ZSTD_isError(zret) && "ZSTD_initCStream failed");

/* Try compressing. */
zret = ZSTD_compressStream(zcs, &zout, &zin);
assert(!ZSTD_isError(zret) && "ZSTD_compressStream failed");

ZSTD_freeCStream(zcs);

return 0;
}
```

Compile and link against a distro-packaged zstd 1.5.7. Run the binary:

```console
$ gcc -g example.c -o example -lzstd && ./example
example: example.c:24: main: Assertion `!ZSTD_isError(zret) && "ZSTD_compressStream failed"' failed.
```

Compression fails.

**Expected behavior**

Compression succeeds.

**Additional context**

The failure occurs on [this line](https://github.com/facebook/zstd/blob/v1.5.7/lib/compress/zstdmt_compress.c#L1331) with `capacity == 0x80000000`, `sectionsSize == 0x20000000`, `mtctx->targetSectionSize == 0x20000000`, `nbWorkers == 1`, `nbSlackBuffers == 3`.

Guida per i contributori

Apri la guida per i contributori

Valutazione

Questa issue non è ancora stata valutata.

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.