dmlc / dmlc/dgl

RDMA Feature Collection Support

Open
#4,161 2 comments 0 reactions 1 assignee Claimed by @Rhett-Ying View on GitHub
feature request
Dominant language
Python
Stars
14.3k
Forks
3.1k
PR merge metrics
No merged PRs in 30d

Description

@VoVAllen @Rhett-Ying

Quiver Team recently released our [RDMA based feature collection solution](https://github.com/quiver-team/quiver-feature) which is about 5x faster than `TensorPipe`'s solution. `quiver_feature.DistTensorPGAS` is a distributed tensor abstraction above heterogeneous memories and it can be easily integrated into DGL under pytorch ecosystem. We've also [released docs about how we use RDMA to get the best performance](https://github.com/quiver-team/quiver-feature/blob/main/docs/rdma_details.md) which might be helpful if you guys want to support RDMA feature collection using TensorPipe.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.