alibaba / alibaba/DataX

otsStreamReader 源数据表发生分区变动后,同步数据报错

Open
#890 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
17.4k
Forks
5.7k
PR merge metrics
No merged PRs in 30d

Description

当ots的表发生分区合并后ots.describeStream api返回的shardList 包含父shareId信息时 otsStreamReader 运行报错;
报错结果如下:
![image](https://user-images.githubusercontent.com/43363049/99910603-f9a23b80-2d29-11eb-90d7-689827d5eb52.png)
![image](https://user-images.githubusercontent.com/43363049/99910762-d88e1a80-2d2a-11eb-950a-e49a6794f981.png)

报错原码位置:
![image](https://user-images.githubusercontent.com/43363049/99910635-1a6a9100-2d2a-11eb-8e37-45b4dd47b73e.png)

ots.describeStream api返回的shardList数据如下:
![image](https://user-images.githubusercontent.com/43363049/99910672-41c15e00-2d2a-11eb-897f-45ebe0470ada.png)

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reading the otsStreamReader code at the reported error location and compare its shardList handling with ots.describeStream responses after a partition merge. Reproduce the parent-shard case and verify that synchronization completes without the reported error.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
databases
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.