mysql到HDFS,kerberos配置报错 Can't get Kerberos realm

报错日志如下，麻烦解惑

2018-07-10 08:10:13.144 [main] INFO VMInfo - VMInfo# operatingSystem class => sun.management.OperatingSystemImpl
2018-07-10 08:10:13.161 [main] INFO Engine - the machine info =>

osInfo: Oracle Corporation 1.8 25.102-b14
jvmInfo: Linux amd64 3.10.0-514.el7.x86_64
cpu num: 4

totalPhysicalMemory: -0.00G
freePhysicalMemory: -0.00G
maxFileDescriptorCount: -1
currentOpenFileDescriptorCount: -1

GC Names [PS MarkSweep, PS Scavenge]

MEMORY_NAME | allocation_size | init_size
PS Eden Space | 512.50MB | 512.50MB
Code Cache | 240.00MB | 2.44MB
Compressed Class Space | 1,024.00MB | 0.00MB
PS Survivor Space | 85.00MB | 85.00MB
PS Old Gen | 1,365.50MB | 1,365.50MB
Metaspace | -0.00MB | 0.00MB

2018-07-10 08:10:13.204 [main] INFO Engine -
{
"content":[
{
"reader":{
"name":"mysqlreader",
"parameter":{
"column":[
"*"
],
"connection":[
{
"jdbcUrl":[
"jdbc:mysql://170.100.104.54:3306/uisftech_test"
],
"table":[
"postal_address"
]
}
],
"password":"******",
"username":"root"
}
},
"writer":{
"name":"hdfswriter",
"parameter":{
"column":[
{
"name":"CONTACT_MECH_ID",
"type":"STRING"
},
{
"name":"TO_NAME",
"type":"STRING"
},
{
"name":"ATTN_NAME",
"type":"STRING"
},
{
"name":"ADDRESS1",
"type":"STRING"
},
{
"name":"ADDRESS2",
"type":"STRING"
},
{
"name":"ADDRESS3",
"type":"STRING"
},
{
"name":"DIRECTIONS",
"type":"STRING"
},
{
"name":"CITY",
"type":"STRING"
},
{
"name":"POSTAL_CODE",
"type":"STRING"
},
{
"name":"POSTAL_CODE_EXT",
"type":"STRING"
},
{
"name":"COUNTRY_GEO_ID",
"type":"STRING"
},
{
"name":"STATE_PROVINCE_GEO_ID",
"type":"STRING"
},
{
"name":"COUNTY_GEO_ID",
"type":"STRING"
},
{
"name":"POSTAL_CODE_GEO_ID",
"type":"STRING"
},
{
"name":"GEO_POINT_ID",
"type":"STRING"
},
{
"name":"LAST_UPDATED_STAMP",
"type":"STRING"
},
{
"name":"LAST_UPDATED_TX_STAMP",
"type":"STRING"
},
{
"name":"CREATED_STAMP",
"type":"STRING"
},
{
"name":"CREATED_TX_STAMP",
"type":"STRING"
}
],
"compress":"gzip",
"defaultFS":"hdfs://170.100.100.95:9000",
"fieldDelimiter":"\t",
"fileName":"dataxtest",
"fileType":"text",
"haveKerberos":"true",
"kerberosKeytabFilePath":"/var/run/cloudera-scm-agent/process/3398-hive-HIVESERVER2/hive.keytab",
"kerberosPrincipal":"hive/bigdatanode01.csebank.com@CSEBANK.COM",
"path":"/user/hive/warehouse/uisftech_test.db/postal_address",
"writeMode":"append"
}
}
}
],
"setting":{
"speed":{
"channel":"2"
}
}
}

2018-07-10 08:10:13.235 [main] WARN Engine - prioriy set to 0, because NumberFormatException, the value is: null
2018-07-10 08:10:13.237 [main] INFO PerfTrace - PerfTrace traceId=job_-1, isEnable=false, priority=0
2018-07-10 08:10:13.238 [main] INFO JobContainer - DataX jobContainer starts job.
2018-07-10 08:10:13.242 [main] INFO JobContainer - Set jobId = 0
2018-07-10 08:10:13.786 [job-0] INFO OriginalConfPretreatmentUtil - Available jdbcUrl:jdbc:mysql://170.100.104.54:3306/uisftech_test?yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true.
2018-07-10 08:10:13.788 [job-0] WARN OriginalConfPretreatmentUtil - 您的配置文件中的列配置存在一定的风险. 因为您未配置读取数据库表的列，当您的表字段个数、类型有变动时，可能影响任务正确性甚至会运行出错。请检查您的配置并作出修改.
2018-07-10 08:10:15.731 [job-0] ERROR JobContainer - Exception when job run
java.lang.IllegalArgumentException: Can't get Kerberos realm
at org.apache.hadoop.security.HadoopKerberosName.setConfiguration(HadoopKerberosName.java:65) ~[hadoop-common-2.7.1.jar:na]
at org.apache.hadoop.security.UserGroupInformation.initialize(UserGroupInformation.java:275) ~[hadoop-common-2.7.1.jar:na]
at org.apache.hadoop.security.UserGroupInformation.setConfiguration(UserGroupInformation.java:311) ~[hadoop-common-2.7.1.jar:na]
at com.alibaba.datax.plugin.writer.hdfswriter.HdfsHelper.kerberosAuthentication(HdfsHelper.java:90) ~[hdfswriter-0.0.1-SNAPSHOT.jar:na]
at com.alibaba.datax.plugin.writer.hdfswriter.HdfsHelper.getFileSystem(HdfsHelper.java:64) ~[hdfswriter-0.0.1-SNAPSHOT.jar:na]
at com.alibaba.datax.plugin.writer.hdfswriter.HdfsWriter$Job.init(HdfsWriter.java:47) ~[hdfswriter-0.0.1-SNAPSHOT.jar:na]
at com.alibaba.datax.core.job.JobContainer.initJobWriter(JobContainer.java:704) ~[datax-core-0.0.1-SNAPSHOT.jar:na]
at com.alibaba.datax.core.job.JobContainer.init(JobContainer.java:304) ~[datax-core-0.0.1-SNAPSHOT.jar:na]
at com.alibaba.datax.core.job.JobContainer.start(JobContainer.java:113) ~[datax-core-0.0.1-SNAPSHOT.jar:na]
at com.alibaba.datax.core.Engine.start(Engine.java:92) [datax-core-0.0.1-SNAPSHOT.jar:na]
at com.alibaba.datax.core.Engine.entry(Engine.java:171) [datax-core-0.0.1-SNAPSHOT.jar:na]
at com.alibaba.datax.core.Engine.main(Engine.java:204) [datax-core-0.0.1-SNAPSHOT.jar:na]
Caused by: java.lang.reflect.InvocationTargetException: null
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_102]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_102]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_102]
at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_102]
at org.apache.hadoop.security.authentication.util.KerberosUtil.getDefaultRealm(KerberosUtil.java:84) ~[hadoop-auth-2.7.1.jar:na]
at org.apache.hadoop.security.HadoopKerberosName.setConfiguration(HadoopKerberosName.java:63) ~[hadoop-common-2.7.1.jar:na]
... 11 common frames omitted
Caused by: sun.security.krb5.KrbException: Cannot locate default realm
at sun.security.krb5.Config.getDefaultRealm(Config.java:1029) ~[na:1.8.0_102]
... 17 common frames omitted
2018-07-10 08:10:15.739 [job-0] INFO StandAloneJobContainerCommunicator - Total 0 records, 0 bytes | Speed 0B/s, 0 records/s | Error 0 records, 0 bytes | All Task WaitWriterTime 0.000s | All Task WaitReaderTime 0.000s | Percentage 0.00%
2018-07-10 08:10:15.742 [job-0] ERROR Engine -

经DataX智能分析,该任务最可能的错误原因是:
com.alibaba.datax.common.exception.DataXException: Code:[Framework-02], Description:[DataX引擎运行过程出错，具体原因请参看DataX运行结束时的错误诊断信息 .]. - java.lang.IllegalArgumentException: Can't get Kerberos realm
at org.apache.hadoop.security.HadoopKerberosName.setConfiguration(HadoopKerberosName.java:65)
at org.apache.hadoop.security.UserGroupInformation.initialize(UserGroupInformation.java:275)
at org.apache.hadoop.security.UserGroupInformation.setConfiguration(UserGroupInformation.java:311)
at com.alibaba.datax.plugin.writer.hdfswriter.HdfsHelper.kerberosAuthentication(HdfsHelper.java:90)
at com.alibaba.datax.plugin.writer.hdfswriter.HdfsHelper.getFileSystem(HdfsHelper.java:64)
at com.alibaba.datax.plugin.writer.hdfswriter.HdfsWriter$Job.init(HdfsWriter.java:47)
at com.alibaba.datax.core.job.JobContainer.initJobWriter(JobContainer.java:704)
at com.alibaba.datax.core.job.JobContainer.init(JobContainer.java:304)
at com.alibaba.datax.core.job.JobContainer.start(JobContainer.java:113)
at com.alibaba.datax.core.Engine.start(Engine.java:92)
at com.alibaba.datax.core.Engine.entry(Engine.java:171)
at com.alibaba.datax.core.Engine.main(Engine.java:204)
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.security.authentication.util.KerberosUtil.getDefaultRealm(KerberosUtil.java:84)
at org.apache.hadoop.security.HadoopKerberosName.setConfiguration(HadoopKerberosName.java:63)
... 11 more
Caused by: KrbException: Cannot locate default realm
at sun.security.krb5.Config.getDefaultRealm(Config.java:1029)
... 17 more
- java.lang.IllegalArgumentException: Can't get Kerberos realm
at org.apache.hadoop.security.HadoopKerberosName.setConfiguration(HadoopKerberosName.java:65)
at org.apache.hadoop.security.UserGroupInformation.initialize(UserGroupInformation.java:275)
at org.apache.hadoop.security.UserGroupInformation.setConfiguration(UserGroupInformation.java:311)
at com.alibaba.datax.plugin.writer.hdfswriter.HdfsHelper.kerberosAuthentication(HdfsHelper.java:90)
at com.alibaba.datax.plugin.writer.hdfswriter.HdfsHelper.getFileSystem(HdfsHelper.java:64)
at com.alibaba.datax.plugin.writer.hdfswriter.HdfsWriter$Job.init(HdfsWriter.java:47)
at com.alibaba.datax.core.job.JobContainer.initJobWriter(JobContainer.java:704)
at com.alibaba.datax.core.job.JobContainer.init(JobContainer.java:304)
at com.alibaba.datax.core.job.JobContainer.start(JobContainer.java:113)
at com.alibaba.datax.core.Engine.start(Engine.java:92)
at com.alibaba.datax.core.Engine.entry(Engine.java:171)
at com.alibaba.datax.core.Engine.main(Engine.java:204)
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.security.authentication.util.KerberosUtil.getDefaultRealm(KerberosUtil.java:84)
at org.apache.hadoop.security.HadoopKerberosName.setConfiguration(HadoopKerberosName.java:63)
... 11 more
Caused by: KrbException: Cannot locate default realm
at sun.security.krb5.Config.getDefaultRealm(Config.java:1029)
... 17 more

at com.alibaba.datax.common.exception.DataXException.asDataXException(DataXException.java:40)
at com.alibaba.datax.core.job.JobContainer.start(JobContainer.java:162)
at com.alibaba.datax.core.Engine.start(Engine.java:92)
at com.alibaba.datax.core.Engine.entry(Engine.java:171)
at com.alibaba.datax.core.Engine.main(Engine.java:204)
Caused by: java.lang.IllegalArgumentException: Can't get Kerberos realm
at org.apache.hadoop.security.HadoopKerberosName.setConfiguration(HadoopKerberosName.java:65)
at org.apache.hadoop.security.UserGroupInformation.initialize(UserGroupInformation.java:275)
at org.apache.hadoop.security.UserGroupInformation.setConfiguration(UserGroupInformation.java:311)
at com.alibaba.datax.plugin.writer.hdfswriter.HdfsHelper.kerberosAuthentication(HdfsHelper.java:90)
at com.alibaba.datax.plugin.writer.hdfswriter.HdfsHelper.getFileSystem(HdfsHelper.java:64)
at com.alibaba.datax.plugin.writer.hdfswriter.HdfsWriter$Job.init(HdfsWriter.java:47)
at com.alibaba.datax.core.job.JobContainer.initJobWriter(JobContainer.java:704)
at com.alibaba.datax.core.job.JobContainer.init(JobContainer.java:304)
at com.alibaba.datax.core.job.JobContainer.start(JobContainer.java:113)
... 3 more
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.security.authentication.util.KerberosUtil.getDefaultRealm(KerberosUtil.java:84)
at org.apache.hadoop.security.HadoopKerberosName.setConfiguration(HadoopKerberosName.java:63)
... 11 more
Caused by: KrbException: Cannot locate default realm
at sun.security.krb5.Config.getDefaultRealm(Config.java:1029)
... 17 more

Start at HdfsHelper.kerberosAuthentication in the hdfswriter plugin and trace the HadoopKerberosName and UserGroupInformation calls shown in the stack trace. Reproduce the failure with the supplied HDFS and Kerberos settings, then define and verify the expected handling or configuration guidance for the missing realm.

Written by the indexing model from the issue text.