Improve handling of imbalanced or small partitions

At the moment, `dask-expr` struggles to deal with imbalanced (https://github.com/coiled/benchmarks/issues/1367#issuecomment-1936080012) or very small (https://github.com/coiled/benchmarks/issues/1381) partitions. We should improve this which likely requires overhauling the Parquet reading to collect better statistics.

Start by examining the Parquet reading path and the two linked benchmark reports about imbalanced and very small partitions. Determine what partition statistics are currently collected and define completion as improved handling of both reported partition cases.

Written by the indexing model from the issue text.