[lake][iceberg] Iceberg does not support union read for primary key table
- Dominant language
- Java
- Stars
- 2.1k
- Forks
- 625
- Avg merge
- 3d 14h
- Merged PRs (30d)
- 97
Description
### Search before asking
- [x] I searched in the [issues](https://github.com/apache/fluss/issues) and found nothing similar.
### Fluss version
0.9.0 (latest release)
### Please describe the bug 🐞
I find Iceberg does not support union read primary key table, IcebergLakeSource#createRecordReader returns IcebergRecordReader which not a subclass of SortedRecordReader, so LakeSnapshotAndLogSplitScanner will fail when pollBatch.
I check `testReadIcebergLakeTable` in `FlinkUnionReadPrimaryKeyTableITCase`, seems it just read records from $lake, but not the union read.
Is there something I'm misunderstanding?
### Solution
_No response_
### Are you willing to submit a PR?
- [ ] I'm willing to submit a PR!
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with IcebergLakeSource#createRecordReader and compare IcebergRecordReader with the expectations of LakeSnapshotAndLogSplitScanner during pollBatch. Inspect testReadIcebergLakeTable in FlinkUnionReadPrimaryKeyTableITCase and verify whether it exercises union reads; done means union reading a primary-key Iceberg table succeeds without the scanner failure.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- data-engineering
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 52/100