NVIDIA/nccl-tests

View on GitHub

NCCL Tests

Stars: 1.7k
Forks: 406
Open beginner issues: 0
Indexed issues: 146

Dominant language: Cuda
License: BSD-3-Clause
Last GitHub push: Aug 28, 2026
Latest indexed: Sep 19, 2026
Contributing guide: No contributing guide
Code of conduct: No code of conduct
Beginner labels: No beginner labels indexed
PR merge metrics: No merged PRs in 30d

Beginner friendly Everything

Open All

146 open issues indexed

"comm_ops.cu", line 25: error: cannot open source file "curand.h" Open

Difficulty 4/5 3-5 days Newbie friendliness 48/100

NVIDIA/nccl-tests#395 · 3 comments · Opened Sep 10, 2026
Will nccl-test support testing for NCCL EP? Open

Difficulty 5/5 Over a week Newbie friendliness 25/100

NVIDIA/nccl-tests#390 · 2 comments · Opened Jul 30, 2026
8xB200 nccl test performance validation Open

Difficulty 4/5 3-5 days Newbie friendliness 35/100

NVIDIA/nccl-tests#389 · Opened Jul 22, 2026
Suggestion to adjust license wording for consistency with standard BSD-style phrasing Open

Difficulty 1/5 Under an hour Newbie friendliness 68/100

NVIDIA/nccl-tests#385 · 1 comment · Opened May 29, 2026
Why are different buffers used for each iteration when `-n` > 1 in nccl-tests? Open

Difficulty 3/5 1-2 days Newbie friendliness 38/100

NVIDIA/nccl-tests#384 · 1 comment · Opened May 29, 2026
alltoall: NCCL_WIN_COLL_SYMMETRIC incorrectly triggers symk GIN init, causing AlltoAll DevComm GIN connection to be silently skipped Open

Difficulty 4/5 3-5 days Newbie friendliness 42/100

NVIDIA/nccl-tests#381 · 6 comments · Opened May 13, 2026
NCCL all-reduce bandwidth drops in VMs when using more than 2 passthrough ConnectX-7 NICs Open

Difficulty 4/5 3-5 days Newbie friendliness 35/100

NVIDIA/nccl-tests#380 · 1 comment · Opened Apr 30, 2026
NCCL_IB_DISABLE=1 not work Open

Difficulty 4/5 3-5 days Newbie friendliness 25/100

NVIDIA/nccl-tests#373 · 2 comments · Opened Feb 25, 2026
h100 only has 70GB on bus Open

Difficulty 4/5 3-5 days Newbie friendliness 25/100

NVIDIA/nccl-tests#371 · 1 comment · Opened Feb 7, 2026
running multi-node with Exclusive Process Open

Difficulty 4/5 3-5 days Newbie friendliness 35/100

NVIDIA/nccl-tests#369 · 18 comments · Opened Jan 21, 2026
Is it applicable to the L20 card? Open

Difficulty 4/5 3-5 days Newbie friendliness 20/100

NVIDIA/nccl-tests#368 · 4 comments · Opened Jan 20, 2026
H20-3e 400Gx8 Roce Actual performance experimental data Open

Difficulty 5/5 Over a week Newbie friendliness 20/100

NVIDIA/nccl-tests#367 · Opened Jan 10, 2026
ALL_Reduce_Perf with NVLS in one node(8GPUs) Open

Difficulty 4/5 3-5 days Newbie friendliness 25/100

NVIDIA/nccl-tests#366 · 1 comment · Opened Jan 9, 2026
Segfault while running benchmarks Open

Difficulty 5/5 Over a week Newbie friendliness 25/100

NVIDIA/nccl-tests#363 · Opened Dec 12, 2025
Low NCCL Performance in ubuntu kernel 6.8.x Open

Difficulty 4/5 3-5 days Newbie friendliness 35/100

NVIDIA/nccl-tests#362 · 6 comments · Opened Dec 10, 2025
Multi-node all_reduce_perf / alltoall_perf failures: socketFinalizeAccept wrong type 4 != 3 over TCP and IB QP invalid request errors Open

Difficulty 5/5 Over a week Newbie friendliness 20/100

NVIDIA/nccl-tests#360 · 4 comments · Opened Nov 24, 2025
Does the GinAlltoAllKernel have send/receive conflicts in in-place mode? Open

Difficulty 4/5 3-5 days Newbie friendliness 25/100

NVIDIA/nccl-tests#357 · 1 comment · Opened Oct 30, 2025
test deviceImpl And got error 'an illegal memory access was encountered' Open

Difficulty 4/5 3-5 days Newbie friendliness 25/100

NVIDIA/nccl-tests#355 · 5 comments · Opened Oct 30, 2025
Latest Commits Causing Compile Errors for File alltoall.cu Open

Difficulty 3/5 1-2 days Newbie friendliness 35/100

NVIDIA/nccl-tests#354 · 2 comments · 4 reactions · Opened Oct 27, 2025
can ncu profiling peer memory traffic Open

Difficulty 4/5 3-5 days Newbie friendliness 25/100

NVIDIA/nccl-tests#348 · Opened Oct 10, 2025
alltoall not in generate.py Open

Difficulty 2/5 1-3 hours Newbie friendliness 45/100

NVIDIA/nccl-tests#344 · 1 comment · Opened Sep 10, 2025
can alltoall use zero copy (only send without recv) by ncclCommRegister? Open

Difficulty 5/5 Over a week Newbie friendliness 25/100

NVIDIA/nccl-tests#341 · 6 comments · Opened Aug 27, 2025
Some benchmarks fail with PXN enabled Open

Difficulty 4/5 3-5 days Newbie friendliness 35/100

NVIDIA/nccl-tests#340 · Opened Aug 19, 2025
UnknownError when trying to profile NCCL all reduce perf test with Nsight Compute Open

Difficulty 4/5 3-5 days Newbie friendliness 35/100

NVIDIA/nccl-tests#339 · Opened Aug 17, 2025
Can --agg_iters=10 and --blocking=1 be used simultaneously during testing? Open

Difficulty 4/5 3-5 days Newbie friendliness 25/100

NVIDIA/nccl-tests#337 · Opened Aug 8, 2025
seg fault when running test on AWS EC2 G5 Open

Difficulty 4/5 3-5 days Newbie friendliness 25/100

NVIDIA/nccl-tests#336 · Opened Jul 28, 2025
5090 bandwith is 0 Open

Difficulty 4/5 3-5 days Newbie friendliness 20/100

NVIDIA/nccl-tests#335 · Opened Jul 25, 2025
5090 alltoall bandwith is 0 Open

Difficulty 4/5 3-5 days Newbie friendliness 20/100

NVIDIA/nccl-tests#334 · 5 comments · Opened Jul 25, 2025
How to run `nccl-test` with NCCL 2.27 to benchmark symmetric memory? Open

Difficulty 4/5 3-5 days Newbie friendliness 35/100

NVIDIA/nccl-tests#333 · 41 comments · 3 reactions · Opened Jul 19, 2025
Intra-node NCCL P2P bandwidth on A100 with NVLink 3.0 limited to ~72 GB/s (vs 100 GB/s theoretical) Open

Difficulty 5/5 Over a week Newbie friendliness 20/100

NVIDIA/nccl-tests#331 · 2 comments · Opened Jul 14, 2025
Inconsistent Kernel Execution Time in GPU SendRecv Operation Open

Difficulty 5/5 Over a week Newbie friendliness 25/100

NVIDIA/nccl-tests#330 · Opened Jul 13, 2025
Running nccl-tests on two docker on same host - uses same rank 0 Open

Difficulty 5/5 Over a week Newbie friendliness 20/100

NVIDIA/nccl-tests#329 · 1 comment · Opened Jul 1, 2025
NCCL RING Crash on Odd Number of Nodes in Virtualized RoCEv2 Environment Open

Difficulty 4/5 3-5 days Newbie friendliness 30/100

NVIDIA/nccl-tests#328 · 3 comments · Opened Jun 30, 2025
all gather / all reduce showing unidirectional bandwidth peak performance Open

Difficulty 4/5 3-5 days Newbie friendliness 28/100

NVIDIA/nccl-tests#326 · 2 comments · Opened Jun 23, 2025
NCCL test exception with 10 GPU cards on a single machine Open

Difficulty 4/5 3-5 days Newbie friendliness 30/100

NVIDIA/nccl-tests#324 · Opened Jun 20, 2025
Reducing maxBytes to 27951912277 due to memory limitation Open

Difficulty 4/5 3-5 days Newbie friendliness 25/100

NVIDIA/nccl-tests#322 · 2 comments · Opened Jun 15, 2025
nccl-test performance with doca on 2 or 4 HGX hosts is not good. Open

Difficulty 4/5 3-5 days Newbie friendliness 28/100

NVIDIA/nccl-tests#319 · 5 comments · Opened Jun 11, 2025
Incorrect busbw Calculation in all_reduce.cu with NVLink SHARP Enabled Open

Difficulty 4/5 3-5 days Newbie friendliness 35/100

NVIDIA/nccl-tests#312 · 3 reactions · Opened May 30, 2025
CollNet randomly hangs or fallbacks to P2P Open

more info needed

Difficulty 5/5 Over a week Newbie friendliness 25/100

NVIDIA/nccl-tests#311 · 1 comment · Opened May 26, 2025
Acceptable performance of nccl-test on 2 or 4 HGX hosts? Open

question triaged

Difficulty 5/5 Over a week Newbie friendliness 20/100

NVIDIA/nccl-tests#309 · 12 comments · Opened May 20, 2025
Why is the NCCL performance test low on the RTX 4090? Open

Difficulty 4/5 3-5 days Newbie friendliness 20/100

NVIDIA/nccl-tests#307 · 8 comments · Opened May 9, 2025
Question regarding CUDA gencode support Open

Difficulty 4/5 3-5 days Newbie friendliness 20/100

NVIDIA/nccl-tests#305 · Opened Apr 23, 2025
How to know version number from NCCL test executable file? Open

Difficulty 3/5 1-2 days Newbie friendliness 30/100

NVIDIA/nccl-tests#302 · 1 comment · Opened Apr 8, 2025
Failure when testing on 2-nodes Open

invalid question

Difficulty 4/5 3-5 days Newbie friendliness 25/100

NVIDIA/nccl-tests#299 · 2 comments · Opened Mar 31, 2025
Can the results of nccl-tests be different depending on the version of driver Open

question triaged

Difficulty 4/5 3-5 days Newbie friendliness 35/100

NVIDIA/nccl-tests#297 · 2 comments · Opened Mar 26, 2025
Getting Avg bus bandwidth = 0 when running all_reduce_perf in nccl-tests Open

question triaged

Difficulty 4/5 3-5 days Newbie friendliness 25/100

NVIDIA/nccl-tests#294 · 9 comments · Opened Mar 14, 2025
It is not possible to determine whether this bandwidth value is normal. Open

Difficulty 5/5 Over a week Newbie friendliness 20/100

NVIDIA/nccl-tests#293 · 6 comments · Opened Mar 7, 2025
Single-node and multi-node performance of H100 nodes. Open

Difficulty 4/5 3-5 days Newbie friendliness 25/100

NVIDIA/nccl-tests#290 · 3 comments · Opened Mar 3, 2025
NCCL Test Multi-node Bus Bandwidth Tuning issue Open

Difficulty 5/5 Over a week Newbie friendliness 20/100

NVIDIA/nccl-tests#283 · Opened Jan 18, 2025
nccl-tests cannot perform multi-machine interconnection through RDMA in the docker container. Open

Difficulty 4/5 3-5 days Newbie friendliness 25/100

NVIDIA/nccl-tests#281 · 10 comments · Opened Jan 15, 2025

NVIDIA/nccl-tests

Get new issues in your inbox