ByteDance-Seed / ByteDance-Seed/Triton-distributed
Why is there no performance improvements on H20 GPUs?
- Dominant language
- Python
- Stars
- 1.5k
- Forks
- 172
- PR merge metrics
- No merged PRs in 30d
Description
I test Gemm + ReduceScatter On Nvdia H20 Gpus, Intra-GEMM, 1 Node 8 H20Gpus.
Trition-Distributed is slower than Torch + NCCL.
I have a question : Does Trition-Distributed need a high-performance machine such as H100、H800 or MI300X to get performance improvements?
| MNK | TritonDistributed(ms) | Torch + NCCL(ms) |
| --- | ---| ---|
8192x8192x29568 | 4.17 | 4.01
32x8192x29568 | 0.55 | 0.07
Contributor guide
Research direction
No files or tests are identified in the issue. Start by reproducing the one-node, eight-H20 Gemm plus ReduceScatter measurements for the two listed MNK cases, then compare Triton-Distributed with Torch plus NCCL. Done means establishing whether the gap is expected on H20 hardware and documenting the relevant performance requirements or cause.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- distributed-systems, performance
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100