Voting Parallel Learner
- Dominant language
- C++
- Stars
- 28.8k
- Forks
- 8.9k
- Avg merge
- 1d 12h
- Merged PRs (30d)
- 54
Description
LightGBM implements a voting-parallel tree learner to reduce the communication overhead between nodes for datasets with a large number of features. Currently, I'm working on a project that requests on the order of 2000 features, and we've found that, even with NCCL, the communication is a major component of the fitting time, especially when one scales to more than one machine of 8 GPUs. Is there any plan to support the two-round voting system proposed in the paper?
Currently, XGBoost supports the data-parallel and feature-parallel learning through the `data_split_mode` in DMatrix.
Any pointers to the code or a rough implementation plan would also be appreciated, as I'm not familiar with this codebase.
[LightGBM-a-communication-efficient-parallel-algorithm-for-decision-tree.pdf](https://github.com/user-attachments/files/16154620/LightGBM-a-communication-efficient-parallel-algorithm-for-decision-tree.pdf)
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.