[FEATURE] Assignment strategies under different memory capacity configurations
- Dominant language
- Java
- Stars
- 454
- Forks
- 172
- Avg merge
- 5d 17h
- Merged PRs (30d)
- 5
Description
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
### Search before asking
- [X] I have searched in the [issues](https://github.com/apache/incubator-uniffle/issues?q=is%3Aissue) and found no similar issues.
### Describe the feature
If using the different memory capacity in one uniffle cluster and the underlying disk capacity is same, it will make the large memory capacity machine disk usage relatively high. Sometimes this will make local disk full and fallback to HDFS (it's really slow)
Anyway, I know if the HDFS is slow and local disk is full, this will reflect to the server memory of large memory machine. But this will also make some apps slow and the little memory machines are free when this happened.
I hope the better assignment strategies could be introduced for this problem. And I'm bothered by this. Feel free to describe more for this.
### Motivation
_No response_
### Describe the solution
_No response_
### Additional context
_No response_
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
Contributor guide
Research direction
The issue describes assignment strategies for clusters with different memory capacities and equal disk capacity, but names no files, tests, or entry points. Start by locating the existing assignment strategy implementation and its tests; done should include a defined strategy that balances disk usage without relying prematurely on HDFS.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- distributed-systems
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 20/100