datax执行中抛出com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException异常,异常消息是query timeout
- Dominant language
- Java
- Stars
- 17.4k
- Forks
- 5.7k
- PR merge metrics
- No merged PRs in 30d
Description
报错的堆栈信息如上
【使用场景】mysqlreader向doriswriter迁移数据,单个job的数据量比较大,按分区进行迁移的,(已经拆分了很多,库是pb级别的),测试的job总计在十亿级别的数据,拆分后的task单次迁移的数据设置阈值在8千万以内,报错堆栈如下:

【初步分析】

查看源码,这块的异常sql状态码是42开头的,对应sql的queryTimeout未修改,是datax源码里的172800s, socketTime设置的600s(也设置过更高的,仍然会报这个错误),检查了很多超时的设置,就不知道为啥会抛出这错误了,很奇怪;
Contributor guide
No contributing guide indexed for this repository
Research direction
Inspect the timeout handling used by mysqlreader and doriswriter, beginning with the reported 172800s queryTimeout and 600s socketTime settings. Reproduce the large, partitioned migration and compare the exception's SQL state with the effective timeout values; done means identifying why the query timeout is raised or documenting the missing configuration.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java, mysql
- Domain
- data-engineering, databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100