AbsaOSS / AbsaOSS/spark-partition-sizing
Add skew suppression in `DataFramePartitioner`
未关闭
enhancement
- 主要语言
- Scala
- 星标
- 9
- 派生
- 2
- PR 合并指标
- 30 天内没有已合并 PR
描述
## Background
The partitions can be within the required limits provided to `DataFramePartitioner` repatriating methods. But they can still be very non-uniform in size.
## Feature
Add the possibility to suppress such size skews when the repartitions are requested.
## Proposed Solution
Solution Ideas:
1. Seems _standard variation_ would be a good measure for that.
贡献指南
这个仓库没有索引到贡献指南
评估
这个 Issue 还没有评估数据。