deepseek-ai / deepseek-ai/DeepEP
low latency test error with "dispatch_use_fp8 and round_scale and use_ue8m0"
- Dominant language
- Cuda
- Stars
- 10.1k
- Forks
- 1.4k
- Avg merge
- 4d 1h
- Merged PRs (30d)
- 2
Description
In file DeepEP/tests/test_low_latency.py, I added the code (Line 79-83 ):
73 # Check received data
74 if current_x is not x_pure_rand:
75 recv_x = recv_x[:num_valid_tokens]
76 recv_x_amin = recv_x[:, :-128].amin(dim=-1)
77 recv_src_info = recv_src_info[:num_valid_tokens]
78 assert torch.equal(recv_x_amin, recv_x[:, :-128].amax(dim=-1))
**79 if dispatch_use_fp8 and round_scale and use_ue8m0 :
80 recv_x_last = recv_x[:, -1]
81 if (recv_x_last < -1).any():
82 print(round_scale,use_ue8m0)
83 print(f"rank {rank}, num_times {num_times}, expert_id: {expert_id}, recv_x_last contains values less than -1!, nu m_valid_tokens: {num_valid_tokens}, recv_x_last: {recv_x_last}")**
When running the test with dispatch_use_fp8=True, round_scale=True, and use_ue8m0=True on GB200, the condition (recv_x_last < -1).any() is triggered (Line 83 ). This indicates that invalid negative values are appearing in recv_x_last, which should not happen.
Command:python DeepEP/tests/test_low_latency.py --num-processes 4 --num-tokens 768
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.