[FEATURE] Support local storage capacity selector for per-partition
- Dominant language
- Java
- Stars
- 454
- Forks
- 172
- Avg merge
- 5d 17h
- Merged PRs (30d)
- 5
Description
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
### Search before asking
- [X] I have searched in the [issues](https://github.com/apache/incubator-uniffle/issues?q=is%3Aissue) and found no similar issues.
### Describe the feature
Currently, the local storage of per-partition is selected by hash strategy. This is simple but will cause many problems, like many huge partitions will store in the same local disk, if the hash number is same. I know this is rare but it has happened in our internal env.
So it's better to support remaining space based storage selector for per-partition. And I will introduce the pluggable local storage selector, but I'm not sure whether it could support multiple disk storages for one-partition.
Anyway, the pluggable storage selector for localStorageManager is the first step to achieve above.
### Motivation
_No response_
### Describe the solution
_No response_
### Additional context
_No response_
### Are you willing to submit PR?
- [X] Yes I am willing to submit a PR!
Contributor guide
Research direction
Start by locating localStorageManager and the current hash-based per-partition local-storage selection. Clarify how a pluggable selector should choose remaining capacity and whether one partition may use multiple disks; the work is done when the selector contract and supported behavior are specified and covered by tests.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- backend, distributed-systems
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 32/100