deepseek-ai / deepseek-ai/DeepEP

low latency test error with "dispatch_use_fp8 and round_scale and use_ue8m0"

Open
#352 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Cuda
Stars
10.1k
Forks
1.4k
Avg merge
4d 1h
Merged PRs (30d)
2

Description

In file DeepEP/tests/test_low_latency.py, I added the code (Line 79-83 ):

73 # Check received data
74 if current_x is not x_pure_rand:
75 recv_x = recv_x[:num_valid_tokens]
76 recv_x_amin = recv_x[:, :-128].amin(dim=-1)
77 recv_src_info = recv_src_info[:num_valid_tokens]
78 assert torch.equal(recv_x_amin, recv_x[:, :-128].amax(dim=-1))
**79 if dispatch_use_fp8 and round_scale and use_ue8m0 :
80 recv_x_last = recv_x[:, -1]
81 if (recv_x_last < -1).any():
82 print(round_scale,use_ue8m0)
83 print(f"rank {rank}, num_times {num_times}, expert_id: {expert_id}, recv_x_last contains values less than -1!, nu m_valid_tokens: {num_valid_tokens}, recv_x_last: {recv_x_last}")**

When running the test with dispatch_use_fp8=True, round_scale=True, and use_ue8m0=True on GB200, the condition (recv_x_last < -1).any() is triggered (Line 83 ). This indicates that invalid negative values are appearing in recv_x_last, which should not happen.

Command:python DeepEP/tests/test_low_latency.py --num-processes 4 --num-tokens 768

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.