[Improvement] Allow split huge data ShuffleDataFlushEvent to multiple small events
- Dominant language
- Java
- Stars
- 454
- Forks
- 172
- Avg merge
- 5d 17h
- Merged PRs (30d)
- 5
Description
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
### Search before asking
- [X] I have searched in the [issues](https://github.com/apache/incubator-uniffle/issues?q=is%3Aissue) and found no similar issues.
### What would you like to be improved?
[Improvement] Allow split huge data ShuffleDataFlushEvent to multiple small events.
As the problem mentioned by https://github.com/apache/incubator-uniffle/issues/378#issuecomment-1344260350, the small event will benefit more after the PR of #396 is applied.
### How should we improve?
_No response_
### Are you willing to submit PR?
- [X] Yes I am willing to submit a PR!
Contributor guide
Research direction
The issue names ShuffleDataFlushEvent and references issues #378 and #396. Start by reading those discussions and tracing the existing event path; the issue does not identify files, tests, event size limits, or acceptance criteria, so confirm the intended split behavior with maintainers before defining what done means.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- backend, distributed-systems
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100