ByteDance-Seed / ByteDance-Seed/Triton-distributed

nvshmemx_getmem_nbi_block failed when test nvshmem in H800

Open
#43 6 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
Python
Stars
1.5k
Forks
172
PR merge metrics
No merged PRs in 30d

Description

when i compiled from source, it success. But when I run: bash ./launch.sh ./python/triton_dist/test/nvidia/test_nvshmem_api.py to test nvshmem, happend:

Image
when I compile, something happend:

![Image](https://github.com/user-attachments/assets/7127d6c1-80bb-4140-949e-d9e0f78e4cea)
I'm wondering if there is a connection between these two.

And when I run: bash ./launch.sh ./python/triton_dist/test/nvidia/test_gemm_rs.py 8192 8192 29568,the output is:

Image

Contributor guide

Open the contributing guide

Research direction

Reproduce the reported commands on an H800: bash ./launch.sh ./python/triton_dist/test/nvidia/test_nvshmem_api.py and test_gemm_rs.py with the given dimensions. Inspect the build and test output shown in the issue, then trace the failing nvshmem test to determine whether the source-build behavior is related. Done means the test completes successfully or the failure is narrowed to a documented, reproducible cause.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
distributed-systems, testing-qa
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.