microsoft / microsoft/SynapseML
LightGBM and dynamic allocation
Open
area/lightgbm
enhancement
high priority
- Dominant language
- Scala
- Stars
- 5.2k
- Forks
- 868
- Avg merge
- 22h 9m
- Merged PRs (30d)
- 45
Description
Hello,
We are currently trying to use MMLSpark and LightGBM in our Spark cluster. We usually use default dynamic allocation of executors when starting jobs, but LightGBM seems to not work with this setting (the main Thread used to wait for executors connection only replies to the first ones created during the initialisation of the main Spark context).
How is it possible to make LightGBM work in a dynamic executors allocation way?
Thanks!
EDIT: we are using the current `master` branch
Contributor guide
Assessment
This issue has not been assessed yet.