[Improvement] Support combine operation in map stage.
- Dominant language
- Java
- Stars
- 454
- Forks
- 172
- Avg merge
- 5d 17h
- Merged PRs (30d)
- 5
Description
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
### Search before asking
- [X] I have searched in the [issues](https://github.com/apache/incubator-uniffle/issues?q=is%3Aissue) and found no similar issues.
### What would you like to be improved?
When map send buffer to shuffle server, we could reduce data transmission as much as possible through combine operations.
Because most data may be aggregated in the shuffle server, only part of the data can be combined on the map side. , so the proportion of merge may not be very high.
### How should we improve?
_No response_
### Are you willing to submit PR?
- [X] Yes I am willing to submit a PR!
Contributor guide
Research direction
Start by tracing how the map stage sends buffers to the shuffle server and how data is aggregated there. Determine where a combine operation could reduce transmission while preserving shuffle behavior; done means the operation is implemented and its effect is verified with relevant tests or measurements.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- distributed-systems
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100