dmlc / dmlc/xgboost

Voting Parallel Learner

Open
#10,561 1 comment 1 reaction 0 assignees View on GitHub
feature-request
Dominant language
C++
Stars
28.8k
Forks
8.9k
Avg merge
1d 12h
Merged PRs (30d)
54

Description

LightGBM implements a voting-parallel tree learner to reduce the communication overhead between nodes for datasets with a large number of features. Currently, I'm working on a project that requests on the order of 2000 features, and we've found that, even with NCCL, the communication is a major component of the fitting time, especially when one scales to more than one machine of 8 GPUs. Is there any plan to support the two-round voting system proposed in the paper?

Currently, XGBoost supports the data-parallel and feature-parallel learning through the `data_split_mode` in DMatrix.

Any pointers to the code or a rough implementation plan would also be appreciated, as I'm not familiar with this codebase.

[LightGBM-a-communication-efficient-parallel-algorithm-for-decision-tree.pdf](https://github.com/user-attachments/files/16154620/LightGBM-a-communication-efficient-parallel-algorithm-for-decision-tree.pdf)

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.