Help understanding how compaction and normal write work
- Dominant language
- Java
- Stars
- 6.2k
- Forks
- 2.5k
- Avg merge
- 2d 8h
- Merged PRs (30d)
- 111
Description
Hi,
I would like to understand how normal write works after compaction is scheduled.
As we know,during the Hudi compaction schedule phase, Hudi will collect which file slices will be compacted during compaction execution phase.
Before compaction execution phase actullay starts, write operations continue, I would ask where will the write operations write data to after the file slices are marked as to be compacted? to new file groups?
Thanks.
Contributor guide
No contributing guide indexed for this repository
Research direction
The issue names Hudi's compaction schedule and execution phases and asks how normal writes behave after file slices are marked for compaction. No files or tests are identified; start by tracing those phases and the normal write path, and document which file groups receive writes before execution begins.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- data-engineering
- Issue type
- Documentation
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100