How to diagnose NaN predictions?

Hi, I'm trying to understand the root cause for NaN predictions.

Using LightGBMRegressor-trained model in Databricks cluster, I sometimes receive NaNs as predictions.

My progress so far:

I checked trained model dump and found several leaves with "-nan" as value:
```
"left_child":{
"leaf_index":4,
"leaf_value": -nan,
"leaf_weight":34758,
"leaf_count":34758
}
```
```
"right_child":{
"leaf_index":27,
"leaf_value":-nan,
"leaf_weight":4062,
"leaf_count":4062
}
```

(I've reloaded model dump using local python lightgbm 2.3.1, and result is the same. So this must be strictly training-related).

I've tried to repartition dataframe in a couple of different ways, and nans appear for some items in a seemingly random fashion.

**My primary question**: Are there known conditions when a leaf can have 'nan' as value, like value by default?

Second: If this is specific to partitioning, is there a "correct" way to do it?

Third: How to proceed with the investigation? Is it possible to get details about training process?

**Additional info:**

Using library com.microsoft.ml.spark:mmlspark_2.11:1.0.0-rc1

Estimated train size is about 10M, test set about 500K samples.

Target variable is positive, varying from one to several thousand.

Environment:
Azure Databricks, Apache Spark 2.4.5, Scala 2.11