apache / apache/hudi

Need hudi 1.0 jar which is compatiblue for aws emr 7.12 and spark 3.5.6

Open
#18,189 0 comments 0 reactions 0 assignees View on GitHub
type:devtask
Dominant language
Java
Stars
6.2k
Forks
2.5k
Avg merge
2d 8h
Merged PRs (30d)
111

Description

### Task Description

**What needs to be done:**
We are looking for hudi jar which is compatible with emr 7.12 (spark 3.5.6), we need this jar to build our project first and then deploy it in aws emr. But when we look for downloading it from maven repository we could see only 1.0.0-beta2 version of hudi but its having spark 3.4.* which is incompatible with our requirement. Could you please help us to get the right hudi spark bundle jar.
hudi-spark-bundle_2.12-1.0.0.jar

**Why this task is needed:**
Recently aws released emr 7.12 version which supports hudi 1.0.0 version, we are trying to migrate our prject from emr 6.15 (hudi 0.14.0) to 7.12 (hudi 1.0). But we are facing this hudi incompatible issue.

### Task Type

Code improvement/refactoring

### Related Issues

**Parent feature issue:** (if applicable )
**Related issues:**
NOTE: Use `Relationships` button to add parent/blocking issues after issue is created.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by checking the Hudi Maven artifacts and the hudi-spark-bundle_2.12-1.0.0.jar requested in the issue, then review how Hudi builds and publishes Spark bundles. Verify the resulting artifact against AWS EMR 7.12 and Spark 3.5.6; done means the project can build and deploy with a compatible Hudi 1.0 bundle.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, java, spark
Domain
data-engineering, distributed-systems
Issue type
Refactor
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.