dmlc / dmlc/xgboost

Optimization for deep trees on distributed system.

Open
#9,454 0 comments 0 reactions 0 assignees View on GitHub
feature-request
Dominant language
C++
Stars
28.8k
Forks
8.9k
Avg merge
1d 12h
Merged PRs (30d)
54

Description

Currently, XGBoost's performance is sub-optimal when growing deep trees (max_depth > 16) on distributed systems, partly due to memory allocation and partly due to allreduce call. The CPU utilization is relatively low (<60%).

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by profiling distributed training with max_depth greater than 16, focusing on memory allocation, allreduce calls, and CPU utilization. The issue names no files, tests, or entry points, so first identify the relevant deep-tree training and distributed communication paths. Done means a measured improvement in the reported performance bottlenecks.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
distributed-systems, machine-learning, performance
Issue type
Refactor
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.