apache / apache/datafusion

Split Range repartitioning code out of repartition/mod.rs

Open
#24,245 2 comments 0 reactions 1 assignee Claimed by @peterxcli View on GitHub
Dominant language
Rust
Stars
9.3k
Forks
2.4k
Avg merge
3d 7h
Merged PRs (30d)
344

Description

### Is your feature request related to a problem or challenge?

`datafusion/physical-plan/src/repartition/mod.rs` handles repartition execution, spilling, pushdown, protobuf conversion, and the partitioning algorithms in one large module. PR #23854 adds `RangeExpr` and shared Range-routing logic, making the Range-specific code another distinct concern in that file.

### Describe the solution you'd like

Move the Range-specific implementation into a dedicated `repartition/range.rs` submodule, including:

- `RangeExpr`
- the shared Range partition-ID routing helper
- Range-specific partitioner state and focused tests where practical

`BatchPartitioner` and hash-join dynamic filtering should continue to call the same routing implementation. This should be a code-organization-only change with no behavior change.

### Describe alternatives you've considered

Keep the code in `repartition/mod.rs`. That avoids a refactor now, but the module already owns several independent concerns and will be harder to navigate as Range support grows.

### Additional context

Follow-up to #23854, based on [review feedback](https://github.com/apache/datafusion/pull/23854#discussion_r3751585509).

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.