Optimize memory usage with pandas input.
Open
feature-request
performance
- Dominant language
- C++
- Stars
- 28.8k
- Forks
- 8.9k
- Avg merge
- 1d 12h
- Merged PRs (30d)
- 54
Description
- The special `qid` column introduced in https://github.com/dmlc/xgboost/pull/8859 is actually quite expensive, as pandas `drop` method makes a data copy. After some profiling, extracting a dictionary of columns actually saves memory. (reducing about 6GB for 5-fold cv with istella-s)
- We might want to iterate through the columns in C like what we currently do for cuDF.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.