[Feature Request]: Support for s3 as a cache store for InteractiveRunner(Flink) with jupyterlab
- Dominant language
- Java
- Stars
- 8.7k
- Forks
- 4.7k
- Avg merge
- 2d 2h
- Merged PRs (30d)
- 205
Description
### What would you like to happen?
Current only GCS appears to be supported location for distributed cachestore with interactive(flink) runner when using beam from jupterlab. (https://github.com/apache/beam/blob/master/sdks/python/apache_beam/runners/interactive/README.md#status)
it would be helpful to also have support for s3, so that minio can be used as well.
### Issue Priority
Priority: 2 (default / most feature requests should be filed as P2)
### Issue Components
- [ ] Component: Python SDK
- [ ] Component: Java SDK
- [ ] Component: Go SDK
- [ ] Component: Typescript SDK
- [ ] Component: IO connector
- [ ] Component: Beam examples
- [ ] Component: Beam playground
- [ ] Component: Beam katas
- [ ] Component: Website
- [ ] Component: Spark Runner
- [X] Component: Flink Runner
- [ ] Component: Samza Runner
- [ ] Component: Twister2 Runner
- [ ] Component: Hazelcast Jet Runner
- [ ] Component: Google Cloud Dataflow Runner
Contributor guide
Research direction
Start with the InteractiveRunner documentation and the Interactive README section linked in the issue, then trace how the Flink runner configures the distributed cache store and its current GCS location. Done means InteractiveRunner can use an S3-compatible store, including MinIO, and the supported configuration is documented.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- jupyter
- Domain
- cloud, distributed-systems
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100