datax能支持两个带KB认证的hive吗
- Dominant language
- Java
- Stars
- 17.4k
- Forks
- 5.7k
- PR merge metrics
- No merged PRs in 30d
Description
采集一个带KB认证的hive数据到另一个带KB认证的hive中。求大佬指导。
{
"job": {
"setting": {
"speed": {
"channel": 3
},
"errorLimit": {
"record": 0,
"percentage": 0.02
}
},
"content": [
{
"reader": {
"name": "hdfsreader",
"parameter": {
"defaultFS": "hdfs://xxx.xxx.xx.4:8020",
"fileType": "orc",
"path": "/user/hive/warehouse/test.db/test67",
"fileName": "000000_0",
"fieldDelimiter": "\u0001",
"haveKerberos": true,
"kerberosKeytabFilePath": "/etc/keytab/hive.keytab",
"kerberosPrincipal": "hive/control-plane.minikube.internal@TEST.COM",
"nullFormat": "\\N",
"skipHeader": false,
"column": [
{
"index": "0",
"type": "Long"
}
]
}
},
"writer": {
"name": "hdfswriter",
"parameter": {
"defaultFS": "hdfs://xxx.xxx.xxx.5:8020",
"fileType": "orc",
"path": "/user/hive/warehouse/test.db/book",
"fileName": "cw",
"writeMode": "append",
"haveKerberos": true,
"kerberosKeytabFilePath": "/etc/keytab/hive.keytab",
"kerberosPrincipal": "hive/control-plane.minikube.internal@TEST.COM",
"fieldDelimiter": "\u0001",
"nullFormat": "\\N",
"column": [{
"name": "id",
"type": "int"
}]
}
}
}
]
}
}
因为一台机器只能认证一个hive,无法同时连两个KB认证的hive
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with the hdfsreader and hdfswriter entry points in the supplied job configuration, focusing on how each handles Kerberos authentication. Determine whether two independently authenticated Hive/HDFS endpoints can be used in one job, and define the supported configuration and behavior for a successful transfer.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- hadoop, java
- Domain
- backend, databases, distributed-systems
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100