[FEATURE REQUEST]: DataFrame.mapPartitions

I need to merge together a lot of data using a very complicated probabilistic algorithm.
There are a lot of data and it can't be loaded into memory of a one single node.
I need to aggregate data on each partition.
I use `GroupBy(SparkPartitionId()).Apply(...)` however it shuffles the data and significantly decreases performance.
Moreover some partitions goes to the same node because of hash partitioning.
I don't need this shuffling because I sort the data between these `groupBy` calls.

I found out that `mapPartitions` could be the perfect solution.
I just need to gather as much as possible of data in each partition and apply some huge aggregation function to each of them - that's what I want.

Do you have any plans to implement `mapPartitions`?
Is there any other way to get rid of shuffling?