datax向hbase同步数据报错:com.alibaba.datax.common.exception.DataXException: Code:[Hbasewriter-12], Description:[获取hbase BufferedMutator 时出错.]. - org.apache.hadoop.hbase.client.RetriesExhaustedException: Can't get the locations
- Dominant language
- Java
- Stars
- 17.4k
- Forks
- 5.7k
- PR merge metrics
- No merged PRs in 30d
Description
数据量6000的导入没有问题,导入数据7万条时,程序报错,错误内容如下:
```ERROR Engine -
经DataX智能分析,该任务最可能的错误原因是:
com.alibaba.datax.common.exception.DataXException: Code:[Hbasewriter-12], Description:[获取hbase BufferedMutator 时出错.]. - org.apache.hadoop.hbase.client.RetriesExhaustedException: Can't get the locations
at org.apache.hadoop.hbase.client.RpcRetryingCallerWithReadReplicas.getRegionLocations(RpcRetryingCallerWithReadReplicas.java:312)
at org.apache.hadoop.hbase.client.ScannerCallableWithReplicas.call(ScannerCallableWithReplicas.java:155)
at org.apache.hadoop.hbase.client.ScannerCallableWithReplicas.call(ScannerCallableWithReplicas.java:59)
at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithoutRetries(RpcRetryingCaller.java:200)
at org.apache.hadoop.hbase.client.ClientScanner.call(ClientScanner.java:320)
at org.apache.hadoop.hbase.client.ClientScanner.nextScanner(ClientScanner.java:295)
at org.apache.hadoop.hbase.client.ClientScanner.initializeScannerInConstruction(ClientScanner.java:160)
at org.apache.hadoop.hbase.client.ClientScanner.(ClientScanner.java:155)
at org.apache.hadoop.hbase.client.HTable.getScanner(HTable.java:821)
at org.apache.hadoop.hbase.MetaTableAccessor.fullScan(MetaTableAccessor.java:602)
at org.apache.hadoop.hbase.MetaTableAccessor.tableExists(MetaTableAccessor.java:366)
at org.apache.hadoop.hbase.client.HBaseAdmin.tableExists(HBaseAdmin.java:392)
at com.alibaba.datax.plugin.writer.hbase11xwriter.Hbase11xHelper.checkHbaseTable(Hbase11xHelper.java:186)
at com.alibaba.datax.plugin.writer.hbase11xwriter.Hbase11xHelper.getBufferedMutator(Hbase11xHelper.java:92)
at com.alibaba.datax.plugin.writer.hbase11xwriter.HbaseAbstractTask.(HbaseAbstractTask.java:38)
at com.alibaba.datax.plugin.writer.hbase11xwriter.NormalTask.(NormalTask.java:27)
at com.alibaba.datax.plugin.writer.hbase11xwriter.Hbase11xWriter$Task.init(Hbase11xWriter.java:58)
at com.alibaba.datax.core.taskgroup.runner.WriterRunner.run(WriterRunner.java:44)
at java.lang.Thread.run(Thread.java:745)
- org.apache.hadoop.hbase.client.RetriesExhaustedException: Can't get the locations
at org.apache.hadoop.hbase.client.RpcRetryingCallerWithReadReplicas.getRegionLocations(RpcRetryingCallerWithReadReplicas.java:312)
at org.apache.hadoop.hbase.client.ScannerCallableWithReplicas.call(ScannerCallableWithReplicas.java:155)
at org.apache.hadoop.hbase.client.ScannerCallableWithReplicas.call(ScannerCallableWithReplicas.java:59)
at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithoutRetries(RpcRetryingCaller.java:200)
at org.apache.hadoop.hbase.client.ClientScanner.call(ClientScanner.java:320)
at org.apache.hadoop.hbase.client.ClientScanner.nextScanner(ClientScanner.java:295)
at org.apache.hadoop.hbase.client.ClientScanner.initializeScannerInConstruction(ClientScanner.java:160)
at org.apache.hadoop.hbase.client.ClientScanner.(ClientScanner.java:155)
at org.apache.hadoop.hbase.client.HTable.getScanner(HTable.java:821)
at org.apache.hadoop.hbase.MetaTableAccessor.fullScan(MetaTableAccessor.java:602)
at org.apache.hadoop.hbase.MetaTableAccessor.tableExists(MetaTableAccessor.java:366)
at org.apache.hadoop.hbase.client.HBaseAdmin.tableExists(HBaseAdmin.java:392)
at com.alibaba.datax.plugin.writer.hbase11xwriter.Hbase11xHelper.checkHbaseTable(Hbase11xHelper.java:186)
at com.alibaba.datax.plugin.writer.hbase11xwriter.Hbase11xHelper.getBufferedMutator(Hbase11xHelper.java:92)
at com.alibaba.datax.plugin.writer.hbase11xwriter.HbaseAbstractTask.(HbaseAbstractTask.java:38)
at com.alibaba.datax.plugin.writer.hbase11xwriter.NormalTask.(NormalTask.java:27)
at com.alibaba.datax.plugin.writer.hbase11xwriter.Hbase11xWriter$Task.init(Hbase11xWriter.java:58)
at com.alibaba.datax.core.taskgroup.runner.WriterRunner.run(WriterRunner.java:44)
at java.lang.Thread.run(Thread.java:745)
at com.alibaba.datax.common.exception.DataXException.asDataXException(DataXException.java:40)
at com.alibaba.datax.plugin.writer.hbase11xwriter.Hbase11xHelper.getBufferedMutator(Hbase11xHelper.java:102)
at com.alibaba.datax.plugin.writer.hbase11xwriter.HbaseAbstractTask.(HbaseAbstractTask.java:38)
at com.alibaba.datax.plugin.writer.hbase11xwriter.NormalTask.(NormalTask.java:27)
at com.alibaba.datax.plugin.writer.hbase11xwriter.Hbase11xWriter$Task.init(Hbase11xWriter.java:58)
at com.alibaba.datax.core.taskgroup.runner.WriterRunner.run(WriterRunner.java:44)
at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.hadoop.hbase.client.RetriesExhaustedException: Can't get the locations
at org.apache.hadoop.hbase.client.RpcRetryingCallerWithReadReplicas.getRegionLocations(RpcRetryingCallerWithReadReplicas.java:312)
at org.apache.hadoop.hbase.client.ScannerCallableWithReplicas.call(ScannerCallableWithReplicas.java:155)
at org.apache.hadoop.hbase.client.ScannerCallableWithReplicas.call(ScannerCallableWithReplicas.java:59)
at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithoutRetries(RpcRetryingCaller.java:200)
at org.apache.hadoop.hbase.client.ClientScanner.call(ClientScanner.java:320)
at org.apache.hadoop.hbase.client.ClientScanner.nextScanner(ClientScanner.java:295)
at org.apache.hadoop.hbase.client.ClientScanner.initializeScannerInConstruction(ClientScanner.java:160)
at org.apache.hadoop.hbase.client.ClientScanner.(ClientScanner.java:155)
at org.apache.hadoop.hbase.client.HTable.getScanner(HTable.java:821)
at org.apache.hadoop.hbase.MetaTableAccessor.fullScan(MetaTableAccessor.java:602)
at org.apache.hadoop.hbase.MetaTableAccessor.tableExists(MetaTableAccessor.java:366)
at org.apache.hadoop.hbase.client.HBaseAdmin.tableExists(HBaseAdmin.java:392)
at com.alibaba.datax.plugin.writer.hbase11xwriter.Hbase11xHelper.checkHbaseTable(Hbase11xHelper.java:186)
at com.alibaba.datax.plugin.writer.hbase11xwriter.Hbase11xHelper.getBufferedMutator(Hbase11xHelper.java:92)
... 5 more```
Contributor guide
No contributing guide indexed for this repository
Research direction
The failure path is named in the stack trace: Hbase11xHelper.checkHbaseTable and getBufferedMutator, followed by HBase client region-location calls. Start by reproducing the 70,000-row import and inspect the HBase writer and client configuration around that path. Done means identifying the cause of the region-location failure and verifying a resolution or documenting the required configuration.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100