Triage MOR flaky tests due to small file management
- Dominant language
- Java
- Stars
- 6.2k
- Forks
- 2.5k
- Avg merge
- 2d 8h
- Merged PRs (30d)
- 111
Description
In MOR table tests, due to small file handling, we end up expanding existing file group to pack both updates and inserts and hence no log files were generated. but tests expects compaction to happen after 5 commits which fails the test in such cases.
Already made a fix to some of the tests: [https://github.com/apache/hudi/pull/7720/]
but we might need to triage and fix any other similar tests.
one thing we want to ensure is, we do have explicit small file handling tests for MOR table so that disabling at other places is ok.
## JIRA info
- Link: https://issues.apache.org/jira/browse/HUDI-5595
- Type: Test
- Epic: https://issues.apache.org/jira/browse/HUDI-4302
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by reviewing the MOR table tests and the changes in PR 7720. Find other tests that assume compaction after five commits, then verify that explicit small-file handling coverage remains for MOR tables and that the affected tests no longer depend on log files being generated.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- data-engineering, testing
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100