apache / apache/hudi

[Trino] Support partial-update (IS_PARTIAL) log block reads for ordering merge modes

Open
#19,413 0 comments 0 reactions 1 assignee Claimed by @voonhous View on GitHub
Dominant language
Java
Stars
6.2k
Forks
2.5k
Avg merge
2d 8h
Merged PRs (30d)
111

Description

Follow-up from the #19295 review (https://github.com/apache/hudi/pull/19295#discussion_r3679361621).

`HudiTrinoReaderContext.getRecordMerger` returns `HoodieAvroRecordMerger` for EVENT_TIME_ORDERING and `OverwriteWithLatestMerger` for COMMIT_TIME_ORDERING. When a log block carries the `IS_PARTIAL` header, `BufferedRecordMergerFactory.create` wraps the record merger in `PartialUpdateBufferedRecordMerger`, which delegates to `HoodieRecordMerger.partialMerge`. Neither Avro merger overrides `partialMerge`, so reading such a table through the Trino connector hits the throwing default (`UnsupportedOperationException`).

This is not Trino-specific: `HoodieAvroReaderContext` returns the same mergers, and the only production merger implementing `partialMerge` is `DefaultSparkRecordMerger`, so the whole Avro read path shares the gap.

Scope:
- implement `partialMerge` for the Avro mergers in hudi-common (or an Avro partial-merge fallback)
- add a Trino read test over a MoR table with `IS_PARTIAL` log blocks for both ordering modes

Tracked inline via a TODO in `HudiTrinoReaderContext.getRecordMerger`.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.