Blosc / Blosc/c-blosc2

Slowness at DGX, server CPUs (Intel Xeon Platinum 8480C, 8570)

Open
#784 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
C
Stars
584
Forks
112
PR merge metrics
No merged PRs in 30d

Description

Hi Blosc Team,

I have observed significant degradation (2-5 times slower than workstation) in terms of compress/decompress speed at DGXs (CPUs are Intel Xeon Platinum 8480C, 8570). When I reduced the number of threads, it helps, but according the doc in the below, it should be similar speed. But it might be limited since the test size are small, 32 (blocks), 64 (chunks).

https://github.com/Blosc/c-blosc2/blob/6c16487c76895543c001e2e18605fd56c9c444d9/README_THREADED.rst?plain=1#L9

Or.. the current roadmap for 3.0, is exactly for optimization for DGX servers? I found the below points are very relevant.

https://github.com/Blosc/c-blosc2/blob/6c16487c76895543c001e2e18605fd56c9c444d9/ROADMAP-TO-3.0.rst?plain=1#L11
https://github.com/Blosc/c-blosc2/blob/6c16487c76895543c001e2e18605fd56c9c444d9/ROADMAP-TO-3.0.rst#L13

For now, if you can clarify that blosc2 speed (compress/decompress) at DGX is slower than the speed at the most modern workstations, that will be super helpful, and good to know information. Plus, your advice to optimize the latency with the current blosc2 at DGXs.

Btw, it seems obvious that per-core clock speed of Xeon Platinum 8480C, 8570 is weaker than workstation (AMD Threadripper Pro 5975WX).

Image

Contributor guide

Open the contributing guide

Research direction

Start by reading README_THREADED.rst and the linked ROADMAP-TO-3.0.rst sections. Reproduce compress/decompress timings with the reported thread counts and block or chunk sizes on the Intel Xeon Platinum 8480C or 8570, then compare them with a modern workstation. Done means the performance difference and practical DGX optimization guidance are documented.

Written by the indexing model from the issue text.

Assessment

Tech stack
c
Domain
performance
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.