[Feature] Support batch read table without datalake enabled
- Dominant language
- Java
- Stars
- 2.1k
- Forks
- 625
- Avg merge
- 3d 14h
- Merged PRs (30d)
- 97
Description
### Search before asking
- [X] I searched in the [issues](https://github.com/alibaba/fluss/issues) and found nothing similar.
### Motivation
Currently, if a table is not with datalake enabled, Fluss won't support batch read the table. Also, although the table is datalake enabeld, it the lake snapshot doesn't exist, the batch read will also fail which will force users to wait the snapshot available.
We should resolve these two cases for better user experience
### Solution
1: In the case that the table is not with datalake enabeld, if it's primary key table, we should mrege ssts and change logs to read the full data, if it's a log table, we can just read the logs upto the latest offset.
2: In the case that the snapshot is not available for datalake enabeld, we can just consider it as a table without datalake enabeld and then batch read
### Anything else?
_No response_
### Willingness to contribute
- [ ] I'm willing to submit a PR!
Contributor guide
No contributing guide indexed for this repository
Research direction
The issue names no files or tests. Start by locating the batch-read implementation and the existing primary-key, log-table, and datalake snapshot paths. Verify the design for tables without datalake enabled and for missing snapshots, including full primary-key data and log data through the latest offset.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- data-engineering, databases
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100