alibaba / alibaba/DataX

我这要配了,无效,我的集群的nameservices就叫cluster,报错java.lang.IllegalArgumentException: java.net.UnknownHostException: cluster

Open
#197 5 comments 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
17.4k
Forks
5.7k
PR merge metrics
No merged PRs in 30d

Description

"name": "hdfswriter",
"parameter": {
"defaultFS": "hdfs://cluster",
"hadoopConfig": {
"dfs.nameservices": "cluster",
"dfs.ha.namenodes.cluster": "nn1,nn2",
"dfs.namenode.rpc-address.cluster.nn1": "ha01:8020",
"dfs.namenode.rpc-address.cluster.nn2": "ha04:8020",
"dfs.client.failover.proxy.provider.cluster": "org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider"
},
"fieldDelimiter": "\u0001",
"fileName": "data",
"fileType": "orc",
"path": "/apps/hive/warehouse/mlg.db/ad_info_tao",
"writeMode": "truncate",

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with the hdfswriter configuration in the issue, especially defaultFS and the Hadoop HA properties, and reproduce the java.net.UnknownHostException for cluster. Trace how DataX resolves the configured nameservice and compare it with the HDFS client configuration. Done means identifying the configuration mismatch or unsupported setup and documenting a verified working configuration.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
distributed-systems
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.