MOR compact improve
Open
area:table-service
area:writer
component:compaction
from-jira
priority:critical
status:pr-available
type:improvement
- Dominant language
- Java
- Stars
- 6.2k
- Forks
- 2.5k
- Avg merge
- 2d 8h
- Merged PRs (30d)
- 111
Description
In most business scenarios, the latest data is in the latest delta log file, so we sort it from large to small according to the instance time, which can largely avoid rewriting the data in the compact process, and then optimize the compact time
## JIRA info
- Link: https://issues.apache.org/jira/browse/HUDI-3617
- Type: Improvement
Contributor guide
No contributing guide indexed for this repository
Research direction
No files or tests are named. Start by locating the MOR compaction entry point and examining how delta log files are selected and ordered by instance time. Done means the compaction process prioritizes the latest delta log and avoids unnecessary data rewrites while improving compaction time.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- data-engineering, distributed-systems
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100