Flink Iceberg should support Flink's IOs in addition to Iceberg's IO
- Dominant language
- Java
- Stars
- 9.2k
- Forks
- 3.5k
- Avg merge
- 2d 16h
- Merged PRs (30d)
- 129
Description
### Feature Request / Improvement
Flink already has a so called FileSystem implementation for S3, HDFS, and others. Iceberg ships its own implementation via FileIO. In many scenarios, Flink's file system is already setup correctly, but still the user has to go through hoops to setup Iceberg's FileIO.
Flink Iceberg should get a wrapper which discovers Flink's FileSystem and defers calls to it.
### Query engine
Flink
### Willingness to contribute
- [x] I can contribute this improvement/feature independently
- [ ] I would be willing to contribute this improvement/feature with guidance from the Iceberg community
- [ ] I cannot contribute this improvement/feature at this time
Contributor guide
Research direction
Start by reading Iceberg's FileIO abstraction and Flink's FileSystem implementation for S3, HDFS, and other filesystems. Trace how Flink Iceberg currently selects and configures FileIO, then determine how a wrapper can discover and delegate to Flink's FileSystem. Done means users with a correctly configured Flink filesystem no longer need separate Iceberg FileIO setup.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- distributed-systems
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 55/100