facebook / facebook/zstd

ZSTD_insertBtAndGetAllMatches assertion failed

Aperta
#4,610 1 commento 0 reazioni 0 assegnatari Vedi su GitHub
Lingua principale
C
Stelle
27.9k
Fork
2.6k
Merge medio
1g 3h
PR unite (30g)
8

Descrizione

**Describe the bug**
I have a huge file (1300MB) and zstd hits assertion failure randomly when compress that file.

**To Reproduce**
Steps to reproduce the behavior:
1. Downloads data https://kcwu.csie.org/~kcwu/tmp/zstd-crash.o5m.zst and decompress it. The sha256sum of the o5m file is `90746b4adaf1492cb73f4aae80917f32a98be41cb56e09e3b4f5d719d0494726`.
2. Run 'zstd -19 -T0 -f -k -q zstd-crash.o5m' repeatedly.
3. See error
```
zstd: ../lib/compress/zstd_opt.c:722: ZSTD_insertBtAndGetAllMatches: Assertion `memcmp(match, ip, matchLength) == 0' failed.
Caught SIGABRT signal, printing stack:
/lib/x86_64-linux-gnu/libc.so.6(pthread_kill+0x11c) [0x7e840202db2c]
/lib/x86_64-linux-gnu/libc.so.6(gsignal+0x1e) [0x7e8401fd427e]
/lib/x86_64-linux-gnu/libc.so.6(abort+0xdf) [0x7e8401fb78ff]
/lib/x86_64-linux-gnu/libc.so.6(+0x2881b) [0x7e8401fb781b]
/lib/x86_64-linux-gnu/libc.so.6(+0x3b517) [0x7e8401fca517]
zstd(+0x54e7b) [0x5f6d40700e7b]
zstd(+0x5f839) [0x5f6d4070b839]
zstd(+0x182a1) [0x5f6d406c42a1]
zstd(+0x1aae6) [0x5f6d406c6ae6]
zstd(+0x1bda9) [0x5f6d406c7da9]
zstd(+0x6239d) [0x5f6d4070e39d]
zstd(+0x9eb8) [0x5f6d406b5eb8]
/lib/x86_64-linux-gnu/libc.so.6(+0x9caa4) [0x7e840202baa4]
/lib/x86_64-linux-gnu/libc.so.6(+0x129c6c) [0x7e84020b8c6c]
```

I tested the same command repeatedly (with cool down between runs). zstd crashed 3 out of 100 times.

**Expected behavior**
zstd should not crash.

**Desktop (please complete the following information):**
- OS: ubuntu-26.04
- Other relevant hardware specs: AMD Ryzen 7 3700X 8-Core Processor. total memory 128GB. free memory is 36GB when zstd runs.
- zstd is installed from ubuntu package. version: `*** Zstandard CLI (64-bit) v1.5.5, by Yann Collet ***`

**Additional context**
- the file is a static file. No other processes touch the file when zstd compress.
- I stress tested the system and it is stable (memory is good). zstd won't crash with other similar files (repeated run two days).

Guida per i contributori

Apri la guida per i contributori

Valutazione

Questa issue non è ancora stata valutata.

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.