apache / apache/hudi

[SUPPORT] Process Fallout: Unable to upsert due to continues Failure in Compaction

Open
#9,822 6 comments 0 reactions 0 assignees View on GitHub
area:table-service priority:high status:triaged
Dominant language
Java
Stars
6.2k
Forks
2.5k
Avg merge
2d 8h
Merged PRs (30d)
111

Description

**Describe the problem you faced**

Unable to process data into the Hudi tables as inline Compaction is failing.

**To Reproduce**

Previous **hoodie.compact.inline.max.delta.commits** was 6. We changed it to 10.

Hudi MOR Table configuration:
`
{
'hoodie.table.name': asset,
'hoodie.datasource.write.recordkey.field': id,
'hoodie.datasource.write.table.name': asset,
'hoodie.upsert.shuffle.parallelism': 400,
'hoodie.keep.max.commits': 1000,
'hoodie.keep.min.commits': 990,
'hoodie.compact.inline.max.delta.commits': 10,
'hoodie.clean.automatic': 'false',
'hoodie.clean.async': 'false'
'hoodie.datasource.write.operation': 'upsert',
'hoodie.datasource.write.precombine.field': 'cdc_timestamp',
'hoodie.datasource.write.table.type': 'MERGE_ON_READ',
'hoodie.cleaner.policy': 'KEEP_LATEST_COMMITS',
'hoodie.schema.on.read.enable' : "true",
'hoodie.datasource.write.reconcile.schema' : "true",
'hoodie.compact.inline' : "true",
'hoodie.datasource.write.keygenerator.class': 'org.apache.hudi.keygen.NonpartitionedKeyGenerator'
}
`
Error:
`
org.apache.hudi.exception.HoodieCompactionException: Could not compact s3a://*************************/hudi_tables/assets/asset
`

**Environment Description**

* Hudi version : 0.13.0

* Spark version : 3.3.1

* Storage (HDFS/S3/GCS..) : S3

* Running on Docker? (yes/no) : no

**Additional context**

Add any other context about the problem here.

**Stacktrace**

[HudiCompactionIssue20231003.txt](https://github.com/apache/hudi/files/12812602/HudiCompactionIssue20231003.txt)

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reading the attached HudiCompactionIssue20231003.txt stack trace and reviewing the listed Hudi 0.13.0, Spark 3.3.1, S3, and inline-compaction settings. No source file or test is named, so the first step is to identify the failing compaction path and establish a reproducible case; done requires determining why the upsert compaction fails and documenting or testing the fix.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, spark
Domain
data-engineering, distributed-systems
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.