mysql to mysql同步大表速度限制问题
Open
- Dominant language
- Java
- Stars
- 17.4k
- Forks
- 5.7k
- PR merge metrics
- No merged PRs in 30d
Description
因为源库有最大执行时间限制30s,超30秒后SELECT会停止,所以如何才能配置限制SELECT语句的splitPk的单条查询大小,因为源表行数很大,单条SQL查询量都超40W行,查询时间太长会导致`Query execution was interrupted, maximum statement execution time exceeded`错误
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by tracing how DataX's MySQL reader uses splitPk to determine the size of each SELECT, then inspect how statement execution limits are handled. The work would be complete when large-table synchronization can constrain each query enough to avoid the reported 30-second MySQL timeout.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java, mysql
- Domain
- data-engineering, databases
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100