我这要配了,无效,我的集群的nameservices就叫cluster,报错java.lang.IllegalArgumentException: java.net.UnknownHostException: cluster
- Dominant language
- Java
- Stars
- 17.4k
- Forks
- 5.7k
- PR merge metrics
- No merged PRs in 30d
Description
"name": "hdfswriter",
"parameter": {
"defaultFS": "hdfs://cluster",
"hadoopConfig": {
"dfs.nameservices": "cluster",
"dfs.ha.namenodes.cluster": "nn1,nn2",
"dfs.namenode.rpc-address.cluster.nn1": "ha01:8020",
"dfs.namenode.rpc-address.cluster.nn2": "ha04:8020",
"dfs.client.failover.proxy.provider.cluster": "org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider"
},
"fieldDelimiter": "\u0001",
"fileName": "data",
"fileType": "orc",
"path": "/apps/hive/warehouse/mlg.db/ad_info_tao",
"writeMode": "truncate",
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with the hdfswriter configuration in the issue, especially defaultFS and the Hadoop HA properties, and reproduce the java.net.UnknownHostException for cluster. Trace how DataX resolves the configured nameservice and compare it with the HDFS client configuration. Done means identifying the configuration mismatch or unsupported setup and documenting a verified working configuration.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- distributed-systems
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100