Response return in order needs to be redesigned to ensure performance
- Dominant language
- Java
- Stars
- 2.1k
- Forks
- 625
- Avg merge
- 3d 14h
- Merged PRs (30d)
- 97
Description
### Search before asking
- [x] I searched in the [issues](https://github.com/alibaba/fluss/issues) and found nothing similar.
### Fluss version
0.6.0 (latest release)
### Please describe the bug 🐞
In pr #681, we introduced an implementation of response return in order, but during online testing, we found that the `iterator whole exists request mechanism` may impact the performance of write operations when dealing with large volumes of data. Therefore, we need to revert this implementation and consider a new high-performance solution in the future.
### Solution
_No response_
### Are you willing to submit a PR?
- [ ] I'm willing to submit a PR!
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by reading PR #681 and the implementation of the response-return-in-order behavior, especially the iterator whole exists request mechanism. Measure its effect on write performance with large data volumes, then determine how reverting it affects ordered responses. Done requires removing the performance regression while retaining a high-performance ordering solution.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- data-engineering, distributed-systems, performance
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100