[Feature] Support to read the append queue table in order in batch mode
- Dominant language
- Java
- Stars
- 3.4k
- Forks
- 1.4k
- Avg merge
- 1d 11h
- Merged PRs (30d)
- 396
Description
### Search before asking
- [X] I searched in the [issues](https://github.com/apache/paimon/issues) and found nothing similar.
### Motivation
Currently, in batch mode, we do not respect the order of the append queue table. I believe that maintaining the order can be beneficial in certain situations. For instance, if the order is not preserved, each time we batch read from a queue table and write into a first_row table, the results will vary. Therefore, I suggest adding a mode that allows batch reading to also preserve the order within buckets.
### Solution
_No response_
### Anything else?
_No response_
### Are you willing to submit a PR?
- [X] I'm willing to submit a PR!
Contributor guide
No contributing guide indexed for this repository
Research direction
The issue names no files, tests, or entry points. Start by locating batch reads of append queue tables and determine how bucket order is represented; done means a defined mode preserves that order in batch reads and tests verify the behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- data-engineering
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100