Large datasets consumes excessive amounts of memory when fitting ensemble pipelines

When running nyc_taxi.csv dataset through AutoML search (which has ~1.5M rows and 19 columns), it uses up more than 30 GB of memory when fitting an ensembling pipeline. This excessive memory consumption causes AutoML search to crash, which prevents performance test runs from properly finishing.

This issue tracks finding the root cause of this memory consumption and potential fixes to mitigate this.