deepseek-ai / deepseek-ai/DeepEP

[Question] Question on using num_worst_tokens

Open
#505 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
Cuda
Stars
10.1k
Forks
1.4k
Avg merge
4d 1h
Merged PRs (30d)
2

Description

Hi DeepSeek team,

I noticed that `num_worst_tokens` is included in the arguments of dispatch, which makes sense for CUDA Graph support. However, it seems that the values written to `moe_recv_counter` and `moe_recv_expert_counter` are no longer directly accessible. From the code, it looks like these counters can be recomputed from `recv_topk_idx`, since padded entries are set to `-1`.

The [prefill trace](https://github.com/deepseek-ai/profile-data/blob/main/prefill.json) also doesn’t show any H2D memcpy after the dispatch kernel, so I assume you are recomputing those counters inside another kernel?

Would it be possible to write these counters into CUDA tensors directly? For example, open-source projects like SGLang simply copy `num_recv_tokens_per_expert_list` to device memory. Having these values available on CUDA would make it easier to switch to CPU-async mode and may also help avoid redundant memory accesses.

Thanks!

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.