AbsaOSS/spark-partition-sizing

Sizing partitions in Spark

Stars: 9
Forks: 2
Open beginner issues: 0
Indexed issues: 5

Dominant language: Scala
License: Apache-2.0
Last GitHub push: Mar 23, 2026
Latest indexed: Sep 13, 2026
Contributing guide: No contributing guide
Code of conduct: No code of conduct
Beginner labels: No beginner labels indexed
PR merge metrics: No merged PRs in 30d

Open All

5 open issues indexed

Implement `RecordSizer.fromDirectorySize` method Open

AbsaOSS/spark-partition-sizing#6 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 13, 2026
Make `DataTypeSizes` easy to read from the application conifguration Open

enhancement

AbsaOSS/spark-partition-sizing#12 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 13, 2026
Add skew suppression in `DataFramePartitioner` Open

enhancement

AbsaOSS/spark-partition-sizing#13 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 13, 2026
In `DataFramePartitioner.DataFrameFunctions.repartitionByRecordCount` use max instead of average to decide about partitoing Open

enhancement

AbsaOSS/spark-partition-sizing#14 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 13, 2026
`type ByteSize = Long` might not be correct or at least convenient Open

bug

AbsaOSS/spark-partition-sizing#15 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 13, 2026

A short digest of beginner-friendly GitHub issues.