UCC integration.
- Dominant language
- C++
- Stars
- 28.8k
- Forks
- 8.9k
- Avg merge
- 1d 12h
- Merged PRs (30d)
- 54
Description
We have a default implementation of collective in XGBoost that can handle basic communication, but it's not particularly performant as it doesn't utilize state-of-the-art communication patterns. However, the collective module is designed to work with various backends, including NCCL, GRPC, and GRPC-GPU. I propose that we add [UCC](https://github.com/openucx/ucc/tree/master) as an additional option for improved distributed training.
Similar to NCCL integration, we can have a shared linkage with UCC and dlopen simultaneously. We can use the torch integration as a reference implementation.
I haven't looked into the details yet, will update here after 3.1. In the meantime, comments are welcome.
- We can potentially skip the worker connect if UCC is used.
- We can take the chance to create a registry for various backends.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.