apache / apache/hudi

Support reading Lance file format on Hive

Open
#18,557 0 comments 0 reactions 0 assignees View on GitHub
type:devtask
Dominant language
Java
Stars
6.2k
Forks
2.5k
Avg merge
2d 8h
Merged PRs (30d)
111

Description

### Task Description

`HoodieLanceInputFormat` and `HoodieRealtimeInputFormat` are added with skeleton code to allow CREATE TABLE to succeed on Spark. The real implementation is needed for reading Lance file format on Hive. These format classes should route through {@code HoodieFileGroupReaderBasedRecordReader} to get unified MOR log merging, schema evolution, and bootstrap support.

### Task Type

New feature

### Related Issues

**Parent feature issue:** (if applicable )
**Related issues:**
NOTE: Use `Relationships` button to add parent/blocking issues after issue is created.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by locating the skeleton HoodieLanceInputFormat and HoodieRealtimeInputFormat classes and reading HoodieFileGroupReaderBasedRecordReader. Trace how these format classes are used for CREATE TABLE on Spark and how the reader provides MOR log merging, schema evolution, and bootstrap support. Done means Hive can read Lance files through both formats with those capabilities preserved.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
data-engineering
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.