Support reading Lance file format on Hive
- Dominant language
- Java
- Stars
- 6.2k
- Forks
- 2.5k
- Avg merge
- 2d 8h
- Merged PRs (30d)
- 111
Description
### Task Description
`HoodieLanceInputFormat` and `HoodieRealtimeInputFormat` are added with skeleton code to allow CREATE TABLE to succeed on Spark. The real implementation is needed for reading Lance file format on Hive. These format classes should route through {@code HoodieFileGroupReaderBasedRecordReader} to get unified MOR log merging, schema evolution, and bootstrap support.
### Task Type
New feature
### Related Issues
**Parent feature issue:** (if applicable )
**Related issues:**
NOTE: Use `Relationships` button to add parent/blocking issues after issue is created.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by locating the skeleton HoodieLanceInputFormat and HoodieRealtimeInputFormat classes and reading HoodieFileGroupReaderBasedRecordReader. Trace how these format classes are used for CREATE TABLE on Spark and how the reader provides MOR log merging, schema evolution, and bootstrap support. Done means Hive can read Lance files through both formats with those capabilities preserved.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- data-engineering
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100