Bug in DayBasedCompactionStrategy
- Dominant language
- Java
- Stars
- 6.2k
- Forks
- 2.5k
- Avg merge
- 2d 8h
- Merged PRs (30d)
- 111
Description
I believe there is a bug in {{DayBasedCompactionStrategy}} in the method {{filterPartitionPathsByStrategy}} as it returns filtered partition paths in the format {{YYYY/MM/DD}} even when actual partition format is {{{}YYYY-MM-DD{}}}. This means we won’t trigger compaction for {{YYYY-MM-DD}} tables with this strategy.
## JIRA info
- Link: https://issues.apache.org/jira/browse/HUDI-5312
- Type: Bug
---
## Comments
01/Dec/22 18:12;vgudavarthi;cc [~alexey.kudinkin] ;;;
Contributor guide
No contributing guide indexed for this repository
Research direction
Locate DayBasedCompactionStrategy and inspect filterPartitionPathsByStrategy, focusing on how partition dates are formatted and compared. Reproduce the behavior for YYYY-MM-DD partitions and verify that the strategy returns the actual partition format and triggers compaction for those tables.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- data-engineering
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100