facebook / facebook/zstd

Singificant performance regression on large compression level (18) when switching to API v2

Open
#4,587 5 comments 1 reaction 1 assignee Claimed by @Cyan4973 View on GitHub
Dominant language
C
Stars
27.9k
Forks
2.6k
Avg merge
1d 3h
Merged PRs (30d)
8

Description

Hello,

We've recently updated ZSTD from version 1.5.1 to version 1.5.7 and observed a 4x drop in compression speed at level 18 when using the new API. Let me clarify: 1.5.7 itself is fine and doesn't regress against 1.5.1, when the old, deprecaed API is in use. It's the switch to the new API that drops the speed 4x.

Unfortunately I don't have an isolated test case, but the code can be found at:

https://git.picodata.io/core/tarantool/-/commit/392485ed92973ab836b72714a9489b86f68ab9c5

The test is a database benchmark that fills 128KB pages of LSM tree with random data and compresses them with ZSTD. The drop in compression speed when simply switching to the new API is from 40MB/s to 10MB/s on my build box.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.