apache / apache/hudi

Reduce usage of *LogScanner and standardize on *LogReader..

Open
#16,895 2 comments 0 reactions 1 assignee Claimed by @yihua View on GitHub
from-jira priority:blocker status:pr-available type:improvement
Dominant language
Java
Stars
6.2k
Forks
2.5k
Avg merge
2d 8h
Merged PRs (30d)
111

Description

*LogScanner is still used by

* Some paths in Hive
* HoodieMergedReadHandle
* FileSliceReader
* Java Client
* Some odd classes in CLI
* MetadataTable.
* Flink.

How to go about this

- We need a log reader impl, that can work with Avro/Indexed Records, to be used for Java/CLI?
- Need to understand semantic differences between the two classes if any (bridge)
- Any functionality gaps e.g spillable map support?
- Should rest of code simply work with FileGroupReader with optional base file (instead of reader...?)

## JIRA info

- Link: https://issues.apache.org/jira/browse/HUDI-9117
- Type: Improvement
- Epic: https://issues.apache.org/jira/browse/HUDI-7537
- Fix version(s):
- 1.1.0

---

## Comments

07/Mar/25 22:14;vinoth;Look forward to this, [~mansipp] :);;;

---

14/Mar/25 05:34;vinoth;[~mansipp] Ethan informed me that you are focussing on the CLI usages.. So made a subtask for that. cc [~yihua]

I will take up the other usages..;;;

---

14/Mar/25 16:56;yihua;[~vinoth] I pulled in all relevant subtasks we filed before.  You can reorganize them as needed.;;;

---

16/Mar/25 03:23;mansipp;Thanks for creating a subtask [~vinoth]. Yes, I've been focusing on CLI usages. I'll take a look.;;;

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.