dmlc / dmlc/xgboost

Reduce CPU memory usage for `hist` and `approx`.

Open
#7,690 2 comments 0 reactions 0 assignees View on GitHub
feature-request
Dominant language
C++
Stars
28.8k
Forks
8.9k
Avg merge
1d 12h
Merged PRs (30d)
54

Description

The approx tree method is rewritten in 1.6 based on the existing `hist` tree method. Although in general, the performance is better, for non-zero gradient it might consume more memory than before (rmse is fine, the new version uses less memory). I have identified a few bottlenecks:

- [ ] Approx only: the transposed `SparsePage` costs a copy of data. This was present in the old version too due to sketching.
- [ ] The gradient index. Right now, on CPU compression only works with dense data, we might add sparse data support.
- [x] The memory buffer for histograms. This may not look like a significant problem, but the builder requires a double buffer and allocates for all nodes. For the bosch dataset, this can cost about 1.5 GB of memory with depth 8 trees.
- [ ] Hist only: we can avoid column matrix is the input if fully dense.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.