alibaba / alibaba/DataX

datax执行中抛出com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException异常,异常消息是query timeout

Open
#2,262 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
17.4k
Forks
5.7k
PR merge metrics
No merged PRs in 30d

Description

image
报错的堆栈信息如上
【使用场景】mysqlreader向doriswriter迁移数据,单个job的数据量比较大,按分区进行迁移的,(已经拆分了很多,库是pb级别的),测试的job总计在十亿级别的数据,拆分后的task单次迁移的数据设置阈值在8千万以内,报错堆栈如下:
![image](https://github.com/user-attachments/assets/75cc0044-84e2-41ca-80eb-05e2c8112158)
【初步分析】
![image](https://github.com/user-attachments/assets/19a99560-0dc5-4756-97e8-349f82ba521e)
查看源码,这块的异常sql状态码是42开头的,对应sql的queryTimeout未修改,是datax源码里的172800s, socketTime设置的600s(也设置过更高的,仍然会报这个错误),检查了很多超时的设置,就不知道为啥会抛出这错误了,很奇怪;

Contributor guide

No contributing guide indexed for this repository

Research direction

Inspect the timeout handling used by mysqlreader and doriswriter, beginning with the reported 172800s queryTimeout and 600s socketTime settings. Reproduce the large, partitioned migration and compare the exception's SQL state with the effective timeout values; done means identifying why the query timeout is raised or documenting the missing configuration.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, mysql
Domain
data-engineering, databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.