setting useSingleDatasetMode to True gives : java: malloc.c:4033: _int_malloc: Assertion `(unsigned long) (size) >= (unsigned long) (nb)' failed.

setting ```useSingleDatasetMode``` to ```True``` gives
```
java: malloc.c:4033: _int_malloc: Assertion `(unsigned long) (size) >= (unsigned long) (nb)' failed.
```
I keep everything else the same and set that flag to ```False``` and the code runs just fine. At the surface the error looks like some memory overflow but, the single dataset mode was supposed to reduce the memory burden among other things.

Could you please help me understand what is going wrong?

I am using 0.9.4 synapse. My spark version is 3.1.2

A few more details that might help you:

cluster:
1 master, 2 executors - 64 GB, 16 cores each
training data on disk - 1.37 GB

spark-submi config:

spark.executor.memory=10g
spark.executor.instances=10
spark.executor.cores=3
spark.driver.memory=10g
spark.default.parallelism=54
spark.driver.cores=3
spark.driver.memoryOverhead=1024m
spark.executor.memoryOverhead=1024m
spark.dynamicAllocation.enabled=false

model parameters:
```
{
"numIterations":2500,
"learningRate":0.01,
"maxDepth":30,
"earlyStoppingRound":50,
"chunkSize":800000,
"parallelism":"voting_parallel",
"useSingleDatasetMode":true,
"numThreads":14
}
```

Start by reproducing the failure with SynapseML 0.9.4, Spark 3.1.2, the supplied cluster and spark-submit settings, and the model parameters with useSingleDatasetMode enabled. Compare the failing run with the same configuration using the flag disabled; done means identifying the cause of the allocator assertion and documenting or validating a resolution.

Written by the indexing model from the issue text.