Migrate tests using MiniDFSCluster to use external HDFS
- Dominant language
- Java
- Stars
- 6.2k
- Forks
- 2.5k
- Avg merge
- 2d 8h
- Merged PRs (30d)
- 111
Description
To execute unit tests with Java 17, we can't use MiniDFSCluster and have to rely on external HDFS to test with Java 17.
Utils to use external HDFS:
[https://github.com/apache/hudi/blob/2a0223933884cb044e7aa56f205cae926358a030/hudi-common/src/test/java/org/apache/hudi/common/testutils/HoodieTestUtils.java#L241]
## JIRA info
- Link: https://issues.apache.org/jira/browse/HUDI-6601
- Type: Improvement
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with the external-HDFS utility in hudi-common/src/test/java/org/apache/hudi/common/testutils/HoodieTestUtils.java at the linked location, then locate tests that use MiniDFSCluster. Migrate those tests to the external-HDFS approach and verify they can execute with Java 17.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- hadoop, java
- Domain
- distributed-systems, testing-qa
- Issue type
- Refactor
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100