alibaba / alibaba/DataX

hbase20xsqlreader进行splitKey切分结果乱码异常

Open
#994 4 comments 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
17.4k
Forks
5.7k
PR merge metrics
No merged PRs in 30d

Description

**当我在执行phoenix数据同步oracle操作时datax爆出如下异常**
`com.alibaba.datax.common.exception.DataXException: Code:[Hbasewriter-09], Description:[truncate hbase表时发生异常.]. - 查询Phoenix数据出现异常,请检查服务状态或与HBase管理员联系! - com.aliyun.phoenix.shaded.org.apache.calcite.avatica.AvaticaSqlException: Error -1 (00000) : Error while executing SQL "select "LOANACNO","ACFLAG","BAL","PREAMTBAL","PAYOFFDATE","OVERBAL","OVERINTE","OVERAFINE","OVERBFINE","OVERGUAFEE","OVERASSUSHAREAMT","SERIOVERTERM","OVERDUEDATE","OVERDUEDAY" from "LAKE"."YD_LAKE_LOAN" where (';ZK 00023' <= LOANACNO AND LOANACNO < 'JQ|0002s\') ": Remote driver error: RuntimeException: org.apache.phoenix.exception.PhoenixParserException: ERROR 602 (42P00): Syntax error. Missing "LPAREN" at line 1, column 243. -> PhoenixParserException: ERROR 602 (42P00): Syntax error. Missing "LPAREN" at line 1, column 243. -> MissingTokenException: (null exception message)`
**通过查看源码发现datax对phoenix进行如下的主键切分**
`SELECT MIN(LOANACNO),MAX(LOANACNO) FROM LAKE.YD_LAKE_LOAN WHERE (ORGCODE IS NOT NULL AND LOANACNO IS NOT NULL);`
**执行该语句发现结果如下:**
```
+------------------+------------------+
| MIN("LOANACNO") | MAX("LOANACNO") |
+------------------+------------------+
| 00000001 | YGXX000341 |
+------------------+------------------+
```
**本地调试发现切分为如下结果:**
![image](https://user-images.githubusercontent.com/44955013/114645383-f248f200-9d0b-11eb-83e8-6044f10a398a.png)
最终导致数据同步异常

Contributor guide

No contributing guide indexed for this repository

Research direction

The issue identifies the HBase 2.x SQL reader's splitKey calculation and shows the MIN/MAX query plus malformed boundary values. Start by locating that reader's splitKey handling, reproduce the Phoenix synchronization case, and verify that key values remain intact and produce valid SQL boundaries.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, sql
Domain
databases
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.