deepseek-ai / deepseek-ai/DeepEP

Does Deepep's Dispatch/Combine Support Autograd Like PyTorch's all_to_all?

Open
#272 3 comments 0 reactions 0 assignees View on GitHub
Dominant language
Cuda
Stars
10.1k
Forks
1.4k
Avg merge
4d 1h
Merged PRs (30d)
2

Description

Does the Deepep interface Dispatch/Combine have automatic gradient backpropagation functionality similar to torch.distributed.nn.functional.all_to_all? The built-in Autograd feature of torch.distributed.nn.functional.all_to_all automatically propagates the gradients of tensors exchanged during All-to-All communication in the forward pass back to the corresponding GPUs correctly. If the Deepep interface lacks this functionality, what are the possible solutions?

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.