org.apache.hudi.sink.TestWriteCopyOnWrite will failed when local hadoop env exists
- Dominant language
- Java
- Stars
- 6.2k
- Forks
- 2.5k
- Avg merge
- 2d 8h
- Merged PRs (30d)
- 111
Description
org.apache.hudi.exception.HoodieException: Error while checking whether table exists under path:/var/folders/yy/xxx/T/junit8015457800939027663
at org.apache.hudi.util.StreamerUtil.tableExists(StreamerUtil.java:324)
at org.apache.hudi.util.StreamerUtil.initTableIfNotExists(StreamerUtil.java:289)
at org.apache.hudi.util.StreamerUtil.initTableIfNotExists(StreamerUtil.java:276)
at org.apache.hudi.sink.StreamWriteOperatorCoordinator.start(StreamWriteOperatorCoordinator.java:184)
at org.apache.hudi.sink.utils.StreamWriteFunctionWrapper.openFunction(StreamWriteFunctionWrapper.java:133)
at org.apache.hudi.sink.utils.TestWriteBase$TestHarness.preparePipeline(TestWriteBase.java:158)
at org.apache.hudi.sink.utils.TestWriteBase$TestHarness.preparePipeline(TestWriteBase.java:146)
at org.apache.hudi.sink.TestWriteCopyOnWrite.preparePipeline(TestWriteCopyOnWrite.java:449)
at
Caused by: java.net.ConnectException: Call From xxx/127.0.0.1 to localhost:8020 failed on connection exception: java.net.ConnectException: Connection refused; For more details see: http://wiki.apache.org/hadoop/ConnectionRefused
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at org.apache.hadoop.net.NetUtils.wrapWithMessage(NetUtils.java:827)
## JIRA info
- Link: https://issues.apache.org/jira/browse/HUDI-4461
- Type: Test
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with org.apache.hudi.sink.TestWriteCopyOnWrite.preparePipeline and the setup in TestWriteBase$TestHarness.preparePipeline. Trace the StreamWriteOperatorCoordinator.start call into StreamerUtil.tableExists and reproduce the failure with a local Hadoop environment present. Done means the test no longer fails with a connection refusal to localhost:8020 in that environment.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- hadoop, java
- Domain
- testing-qa
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100