[SUPPORT]Hudi Deltastreamer compaction is taking longer duration
Open
area:ingest
area:metadata-table
area:performance
area:table-service
priority:critical
- Dominant language
- Java
- Stars
- 6.2k
- Forks
- 2.5k
- Avg merge
- 2d 8h
- Merged PRs (30d)
- 111
Description
Hi,
I am creating COW table.As per the document compaction is not applicable to COW.
Still compaction is getting triggered.
Usually data write was happening in 2min when ever compaction is getting triggered jobs are staying stuck in running state,
Thanks,
Suneetha
Contributor guide
No contributing guide indexed for this repository
Research direction
The report names Hudi DeltaStreamer, COW tables, and compaction, but no files, tests, or reproduction configuration. Start by checking the COW and compaction behavior in the DeltaStreamer documentation and configuration, then establish whether compaction is incorrectly triggered and whether the write job completes as expected.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- data-engineering, stream-processing
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 20/100