apache / apache/fluss

Response return in order needs to be redesigned to ensure performance

Open
#771 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
2.1k
Forks
625
Avg merge
3d 14h
Merged PRs (30d)
97

Description

### Search before asking

- [x] I searched in the [issues](https://github.com/alibaba/fluss/issues) and found nothing similar.

### Fluss version

0.6.0 (latest release)

### Please describe the bug 🐞

In pr #681, we introduced an implementation of response return in order, but during online testing, we found that the `iterator whole exists request mechanism` may impact the performance of write operations when dealing with large volumes of data. Therefore, we need to revert this implementation and consider a new high-performance solution in the future.

### Solution

_No response_

### Are you willing to submit a PR?

- [ ] I'm willing to submit a PR!

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reading PR #681 and the implementation of the response-return-in-order behavior, especially the iterator whole exists request mechanism. Measure its effect on write performance with large data volumes, then determine how reverting it affects ordered responses. Done requires removing the performance regression while retaining a high-performance ordering solution.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
data-engineering, distributed-systems, performance
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.