deepseek-ai / deepseek-ai/DeepEP

`python test_low_latency.py` blocked in H20

Open
#103 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Cuda
Stars
10.1k
Forks
1.4k
Avg merge
4d 1h
Merged PRs (30d)
2

Description

By analyzing the code, we find the code is blocked in the instantiation of buffer

![Image](https://github.com/user-attachments/assets/126aa226-c40f-483c-9f15-1eac4c96aead)

![Image](https://github.com/user-attachments/assets/ba99c771-ebd9-4474-ab1f-d166b3057df1)

![Image](https://github.com/user-attachments/assets/49ef44f1-e71a-4a6c-8ae4-83574dab9569)

We got the following log when setting "NVSHMEM_DEBUG=INFO"

![Image](https://github.com/user-attachments/assets/5972f4ea-4173-41ba-b98e-a23e78fc1ad2)

What does the Error "cudaErrorInvalidValue 1 cudaErrorInvalidSymbol 13 cudaErrorInvalidMemcpyDirection 21 cudaErrorNoKernelImageForDevice 209" mean? Is it the root cause of process blocking.

Here is the configuration of NVSHMEM
![Image](https://github.com/user-attachments/assets/a9745ff6-de58-47e8-afbf-c8604a5d1f37)

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.