Add HoodieReadClient with java
- Dominant language
- Java
- Stars
- 6.2k
- Forks
- 2.5k
- Avg merge
- 2d 8h
- Merged PRs (30d)
- 111
Description
We might need a hoodie read client in java similar to the one we have for spark.
[Apache Pulsar|https://github.com/apache/pulsar] is doing integration with Hudi, and take Hudi as tiered storage to offload topic cold data into Hudi. When consumers fetch cold data from topic, Pulsar broker will locate the target data is stored in Pulsar or not. If the target data stored in tiered storage (Hudi), Pulsar broker will fetch data from Hudi by Java API, and package them into Pulsar format and dispatch to consumer side.
However, we found current Hudi implementation doesn't support read Hudi table records by Java API, and we couldn't read the target data out from Hudi into Pulsar Broker, which will block the Pulsar & Hudi integration.
h3. What we need
# We need Hudi to support reading records by Java API
# We need Hudi to support read records out which keep the writer order, or support order by specific fields.
## JIRA info
- Link: https://issues.apache.org/jira/browse/HUDI-3892
- Type: Task
- Epic: https://issues.apache.org/jira/browse/HUDI-5197
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by examining the existing Spark read client and the Hudi table-reading APIs. Define the Java API needed by the Pulsar broker, including reading records in writer order or by specified fields. Done means Java callers can read the target Hudi records and the ordering requirement is supported.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- data-engineering
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100