Add Flink support for Blob data type
- Dominant language
- Java
- Stars
- 6.2k
- Forks
- 2.5k
- Avg merge
- 2d 8h
- Merged PRs (30d)
- 111
Description
### Feature Description
Feature Description
(Will split into smaller tickets as we get agreement)
All Flink read/write related classes should handle HoodieSchema Blob type.
- Based on initial scan, it seems Flink ROW/RowType <-> Avro RECORD type handling might already cover reading/writing Blob fields from Flink. Do we need to add more specific handling for exposing the HoodieSchema blob logical type when creating Flink ROW/RowType? Like passing in `"symbols": ["INLINE", "OUT_OF_LINE"]` ?
- Check if we need to add more Integ tests for Flink SQL We should make sure we test with multiple versions of Flink 2.x and for both COW and MOR tables.
- Add functional/integration tests to ensure Flink can read blob data records written by Spark writes, and vice versa.
- Exploration/comparison with Ray based on RFC https://github.com/apache/hudi/blob/master/rfc/rfc-100/rfc-100.md - should we even support (HUDI) Flink with blob if users can integrate with (HUDI) Ray for low-latency use cases?
- Enable read_blob for FlinkSQL for OOL blobs https://github.com/apache/hudi/issues/19032
### User Experience
**How users will use this feature:**
- Configuration changes needed
- API changes
- Usage examples
### Hudi RFC Requirements
**RFC PR link:** (if applicable)
**Why RFC is/isn't needed:**
- Does this change public interfaces/APIs? (Yes/No)
- Does this change storage format? (Yes/No)
- Justification:
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.