Decouple Reader/Writer versioning
- Dominant language
- Java
- Stars
- 6.2k
- Forks
- 2.5k
- Avg merge
- 2d 8h
- Merged PRs (30d)
- 111
Description
Currently Hudi versions both readers and writers w/ single table format version.
This might be disadvantageous in 1:N deployments, where a single writer gets updated but it's not easy to update the readers (in large org deployments for ex). As such, decoupling reader/writer versioning would allow us to introduce a notion of Forward Compatibility where if no features require an upgraded version of the reader then older version of the reader would still be able to read the table by the newer version of the writer
## JIRA info
- Link: https://issues.apache.org/jira/browse/HUDI-5313
- Type: Improvement
Contributor guide
No contributing guide indexed for this repository
Research direction
No files, tests, or entry points are named. Start with the linked JIRA issue HUDI-5313 and trace how the single table format version is defined and consumed; done means reader and writer versions are independently represented and older readers remain usable when no newer reader features are required.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- data-engineering
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100