LightGBM stuck at "reduce at LightGBMClassifier.scala:150"

Nobody has claimed this yet.

I have already noticed the issue https://github.com/Azure/mmlspark/issues/542, but the answer cannot solve my problem.

I have a dataset nearly 72GB and 145 columns. My spark config is
spark-submit \
--master yarn \
--deploy-mode client \
--executor-memory 15g \
--driver-memory 15g \
--executor-cores 1 \
--num-executors 20 \
--packages com.microsoft.ml.spark:mmlspark_2.11:1.0.0-rc1 \
--conf spark.default.parallelism=5000 \
--conf spark.sql.shuffle.partitions=5000 \
--conf spark.dynamicAllocation.enabled=false \
--conf spark.memory.storageFraction=0.3 \
--conf spark.executor.memoryOverhead=15g \
--conf spark.driver.maxResultSize=10g \

if I reduce the dataset size to 24 GB, I could train the model in 40 minutes. But if I increase the dataset to 72GB, the training process would be stuck at "reduce at LightGBMClassifier.scala:150" and report some failed information, "ExecutorLostFailure (executor 9 exited caused by one of the running tasks) Reason: Executor heartbeat timed out after 128370 ms", "java.lang.Exception: Dataset create call failed in LightGBM with error: Socket recv error, code: 104", "java.net.ConnectException: Connection refused"

[AB#1188553](https://msdata.visualstudio.com/b9b2accc-2d1c-45b3-9d24-0eb5d78cc47f/_workitems/edit/1188553)

Start by reading LightGBMClassifier.scala:150 and the reported Spark configuration, then review the executor heartbeat timeout, socket receive error, and connection-refused logs. Compare the successful 24GB run with the stalled 72GB run and determine whether the failure can be reproduced; done means identifying a confirmed cause or a specific configuration or code change to test.

Written by the indexing model from the issue text.