DTStack / DTStack/chunjun

Flinkx读取mysql中途失败,任务却显示成功

Open
#608 2 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
Java
Stars
4.1k
Forks
1.7k
PR merge metrics
No merged PRs in 30d

Description

1.我们的mysql可能有问题(我们还在排查mysql的问题)
2.我们使用mysql to mysql 进行数据同步
3.在yarn上,任务显示正常,但是sink端数据条数和source端数据条数不一致
4.然后用local模式,测试mysql to mysql 发现异常
![image](https://user-images.githubusercontent.com/50513095/154613943-5b7f8906-2468-4041-aae4-dc65e9323817.png)

5.然后我排查flinkx代码发现有个bug,在执行next发生异常时,默认已经把source读取完成了,没有数据可以读取了。然后导致任务成功了。

![image](https://user-images.githubusercontent.com/50513095/154614155-4d407820-6948-47d9-8c05-c0190afed700.png)
![image](https://user-images.githubusercontent.com/50513095/154614215-c94e958d-cb02-46e5-bae9-a99f6d9e4d5f.png)

**效果就是mysql source发生了异常,但是还是导致flinkx任务成功了**

6.我们的修改如下
![image](https://user-images.githubusercontent.com/50513095/154614309-d40df8e1-f222-4b0a-82c1-6a21c6e28731.png)

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.