In spark.XGBoost.trainDistributed, the parameter n_thread, does not seem to take effect most of the time and still only utilizes a single thread
- Dominant language
- C++
- Stars
- 28.8k
- Forks
- 8.9k
- Avg merge
- 1d 12h
- Merged PRs (30d)
- 54
Description
hello,i got a problem when i run xgboost distribute train in yarn. my parameter :
set executor-memory = "32G"
set executor-cores = "8"
set spark.task.cpus = 8
set spark.driver.maxResultSize="24G"
xgboost train param:
set and objective = "binary:logistic"
and num_round = "700"
and missing = "-999.0"
and max_depth= "3"
and num_workers = '1'
and num_early_stopping_rounds = "10"
and n_thread = 8
when task come to this work: **it always 1 running thread, why?** hope for some answers and advice to speed up the trainning
[20:24:42] task 0 got new rank 0
[20:31:41] INFO: /workspace/src/gbm/gbtree.cc:140: Tree method is automatically selected to be 'approx' for distributed training.
[20:31:41] DEBUG: /workspace/src/gbm/gbtree.cc:157: Using tree method: 1
[20:31:41] DEBUG: /workspace/src/gbm/gbtree.cc:122: Using updaters: grow_histmaker
[20:31:41] ======== Monitor (0): TreePruner ========
[20:31:41] INFO: /workspace/src/data/simple_dmatrix.cc:103: Generating new Gradient Index.
[20:35:03] ======== Monitor (0): SortedSketchContainer ========
[20:35:03] AllReduce: 0.017769s, 1 calls @ 17769us
[20:35:03] MakeCuts: 0.02635s, 1 calls @ 26350us
[20:35:03] PushColPage: 22.5197s, 1 calls @ 22519734us
[20:37:06] INFO: /workspace/src/data/simple_dmatrix.cc:103: Generating new Gradient Index.
[20:37:54] ======== Monitor (0): SortedSketchContainer ========
[20:37:54] AllReduce: 0.008843s, 1 calls @ 8843us
[20:37:54] MakeCuts: 0.017436s, 1 calls @ 17436us
[20:37:54] PushColPage: 46.152s, 1 calls @ 46151982us
[20:40:06] INFO: /workspace/src/data/simple_dmatrix.cc:103: Generating new Gradient Index
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.