alibaba / alibaba/DataX

hbase reader 组件当请求超时无限请求问题

Open
#275 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
17.4k
Forks
5.7k
PR merge metrics
No merged PRs in 30d

Description

hbase11xreader插件中Hbase11xReader.java 第77行 fetchOK = this.hbaseTaskProxy.fetchLine(record);

此方法在while true 中调用,异常捕获使用exception 处理所有异常,不是只有脏数据才会抛出异常,如果是第一次请求超时,会无限请求hbase,一直到hbase 宕机且datax进程无法正常退出。这里应该分开捕获特定异常。

Contributor guide

No contributing guide indexed for this repository

Research direction

Start in hbase11xreader's Hbase11xReader.java at line 77, where hbaseTaskProxy.fetchLine(record) is called inside the while loop. Trace which exceptions represent dirty data versus request failures, then verify that a timeout does not cause endless HBase requests and that the DataX process can exit normally.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
backend, databases
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.