apache / apache/hudi

TestHoodieMetadataTableValidator#testAdditionalPartitionsinMDT is flaky

Open
#17,376 1 comment 0 reactions 0 assignees View on GitHub
from-jira priority:high type:devtask
Dominant language
Java
Stars
6.2k
Forks
2.5k
Avg merge
2d 8h
Merged PRs (30d)
111

Description

Refer: https://dev.azure.com/apachehudi/hudi-oss-ci/_build/results?buildId=3425&view=logs&j=d593737e-341b-5ee7-e7ed-3dd84baee240&t=c401e752-aa59-5254-6b0f-d06b8fc601ad

{code:java}
[[ERROR] Tests run: 35, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 255.455 s <<< FAILURE! - in org.apache.hudi.utilities.TestHoodieMetadataTableValidator
[ERROR] testAdditionalPartitionsinMDT{boolean}[2] Time elapsed: 3.158 s <<< ERROR!
org.apache.hudi.exception.HoodieValidationException:
Compare Partitions Failed! Additional 0 partitions from FS, but missing from MDT : "" and additional 1partitions from MDT, but missing from FS listing : "PARTITION3".
at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:114)
at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:86)
at org.junit.platform.launcher.core.DefaultLauncherSession$DelegatingLauncher.execute(DefaultLauncherSession.java:86)
at org.junit.platform.launcher.core.SessionPerRequestLauncher.execute(SessionPerRequestLauncher.java:53)
at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invokeAllTests(JUnitPlatformProvider.java:150)
at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invoke(JUnitPlatformProvider.java:124)
at org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:384)
at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:345)
at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:126)
at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:418)

{code}

## JIRA info

- Link: https://issues.apache.org/jira/browse/HUDI-8991
- Type: Sub-task
- Parent: https://issues.apache.org/jira/browse/HUDI-8695

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with org.apache.hudi.utilities.TestHoodieMetadataTableValidator#testAdditionalPartitionsinMDT and review the referenced Azure CI failure. Reproduce the test repeatedly and trace why the filesystem and metadata-table partition listings differ. Done means the test no longer fails intermittently and continues to validate additional partitions correctly.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
testing
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.