alibaba / alibaba/DataX

sqlserver2mysql 时speed 速度慢慢下降 mysql数据库最后宕机了

Open
#1,618 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
17.4k
Forks
5.7k
PR merge metrics
No merged PRs in 30d

Description

2022-11-30 13:37:36.717 [job-0] INFO StandAloneJobContainerCommunicator - Total 0 records, 0 bytes | Speed 0B/s, 0 records/s | Error 0 records, 0 bytes | All Task WaitWriterTime 0.000s | All Task WaitReaderTime 0.000s | Percentage 0.00%
2022-11-30 13:37:46.721 [job-0] INFO StandAloneJobContainerCommunicator - Total 2560 records, 1028585 bytes | Speed 100.45KB/s, 256 records/s | Error 0 records, 0 bytes | All Task WaitWriterTime 0.001s | All Task WaitReaderTime 0.435s | Percentage 0.00%
2022-11-30 13:37:56.723 [job-0] INFO StandAloneJobContainerCommunicator - Total 4608 records, 1850706 bytes | Speed 80.29KB/s, 204 records/s | Error 0 records, 0 bytes | All Task WaitWriterTime 13.746s | All Task WaitReaderTime 0.666s | Percentage 0.00%
2022-11-30 13:38:06.724 [job-0] INFO StandAloneJobContainerCommunicator - Total 8704 records, 3494546 bytes | Speed 160.53KB/s, 409 records/s | Error 0 records, 0 bytes | All Task WaitWriterTime 28.363s | All Task WaitReaderTime 1.195s | Percentage 0.00%
2022-11-30 13:38:16.730 [job-0] INFO StandAloneJobContainerCommunicator - Total 10752 records, 4319693 bytes | Speed 80.58KB/s, 204 records/s | Error 0 records, 0 bytes | All Task WaitWriterTime 37.749s | All Task WaitReaderTime 1.464s | Percentage 0.00%
2022-11-30 13:38:26.734 [job-0] INFO StandAloneJobContainerCommunicator - Total 12800 records, 5144782 bytes | Speed 80.57KB/s, 204 records/s | Error 0 records, 0 bytes | All Task WaitWriterTime 44.892s | All Task WaitReaderTime 1.758s | Percentage 0.00%
2022-11-30 13:38:36.735 [job-0] INFO StandAloneJobContainerCommunicator - Total 14848 records, 5970042 bytes | Speed 80.59KB/s, 204 records/s | Error 0 records, 0 bytes | All Task WaitWriterTime 52.025s | All Task WaitReaderTime 2.044s | Percentage 0.00%
2022-11-30 13:38:46.736 [job-0] INFO StandAloneJobContainerCommunicator - Total 17888 records, 7194774 bytes | Speed 119.60KB/s, 304 records/s | Error 0 records, 0 bytes | All Task WaitWriterTime 67.081s | All Task WaitReaderTime 2.462s | Percentage 0.00%
2022-11-30 13:38:56.739 [job-0] INFO StandAloneJobContainerCommunicator - Total 18944 records, 7619968 bytes | Speed 41.52KB/s, 105 records/s | Error 0 records, 0 bytes | All Task WaitWriterTime 67.082s | All Task WaitReaderTime 2.610s | Percentage 0.00%
2022-11-30 13:39:06.740 [job-0] INFO StandAloneJobContainerCommunicator - Total 23040 records, 9269992 bytes | Speed 161.13KB/s, 409 records/s | Error 0 records, 0 bytes | All Task WaitWriterTime 85.551s | All Task WaitReaderTime 3.149s | Percentage 0.00%
2022-11-30 13:39:16.742 [job-0] INFO StandAloneJobContainerCommunicator - Total 25088 records, 10094518 bytes | Speed 80.52KB/s, 204 records/s | Error 0 records, 0 bytes | All Task WaitWriterTime 92.539s | All Task WaitReaderTime 3.321s | Percentage 0.00%
2022-11-30 13:39:26.748 [job-0] INFO StandAloneJobContainerCommunicator - Total 27136 records, 10915551 bytes | Speed 80.18KB/s, 204 records/s | Error 0 records, 0 bytes | All Task WaitWriterTime 99.481s | All Task WaitReaderTime 3.604s | Percentage 0.00%
2022-11-30 13:39:36.753 [job-0] INFO StandAloneJobContainerCommunicator - Total 31232 records, 12558113 bytes | Speed 160.41KB/s, 409 records/s | Error 0 records, 0 bytes | All Task WaitWriterTime 114.679s | All Task WaitReaderTime 4.161s | Percentage 0.00%
2022-11-30 13:39:56.756 [job-0] INFO StandAloneJobContainerCommunicator - Total 35328 records, 14204919 bytes | Speed 80.41KB/s, 204 records/s | Error 0 records, 0 bytes | All Task WaitWriterTime 129.636s | All Task WaitReaderTime 4.483s | Percentage 0.00%
2022-11-30 13:40:06.758 [job-0] INFO StandAloneJobContainerCommunicator - Total 37376 records, 15028167 bytes | Speed 80.39KB/s, 204 records/s | Error 0 records, 0 bytes | All Task WaitWriterTime 139.146s | All Task WaitReaderTime 4.762s | Percentage 0.00%
2022-11-30 13:40:16.759 [job-0] INFO StandAloneJobContainerCommunicator - Total 39424 records, 15851508 bytes | Speed 80.40KB/s, 204 records/s | Error 0 records, 0 bytes | All Task WaitWriterTime 149.142s | All Task WaitReaderTime 5.058s | Percentage 0.00%
2022-11-30 13:40:36.763 [job-0] INFO StandAloneJobContainerCommunicator - Total 43520 records, 17498144 bytes | Speed 80.40KB/s, 204 records/s | Error 0 records, 0 bytes | All Task WaitWriterTime 166.683s | All Task WaitReaderTime 5.494s | Percentage 0.00%
2022-11-30 13:40:46.764 [job-0] INFO StandAloneJobContainerCommunicator - Total 47616 records, 19144996 bytes | Speed 160.83KB/s, 409 records/s | Error 0 records, 0 bytes | All Task WaitWriterTime 182.259s | All Task WaitReaderTime 6.075s | Percentage 0.00%
2022-11-30 13:41:06.765 [job-0] INFO StandAloneJobContainerCommunicator - Total 51712 records, 20791919 bytes | Speed 80.42KB/s, 204 records/s | Error 0 records, 0 bytes | All Task WaitWriterTime 198.836s | All Task WaitReaderTime 6.552s | Percentage 0.00%
2022-11-30 13:41:16.770 [job-0] INFO StandAloneJobContainerCommunicator - Total 53760 records, 21615435 bytes | Speed 80.42KB/s, 204 records/s | Error 0 records, 0 bytes | All Task WaitWriterTime 207.513s | All Task WaitReaderTime 6.732s | Percentage 0.00%
2022-11-30 13:41:26.775 [job-0] INFO StandAloneJobContainerCommunicator - Total 55808 records, 22438825 bytes | Speed 80.41KB/s, 204 records/s | Error 0 records, 0 bytes | All Task WaitWriterTime 219.057s | All Task WaitReaderTime 7.016s | Percentage 0.00%
2022-11-30 13:41:36.780 [job-0] INFO StandAloneJobContainerCommunicator - Total 57856 records, 23262169 bytes | Speed 80.40KB/s, 204 records/s | Error 0 records, 0 bytes | All Task WaitWriterTime 229.487s | All Task WaitReaderTime 7.201s | Percentage 0.00%
2022-11-30 13:41:46.785 [job-0] INFO StandAloneJobContainerCommunicator - Total 59904 records, 24085633 bytes | Speed 80.42KB/s, 204 records/s | Error 0 records, 0 bytes | All Task WaitWriterTime 239.642s | All Task WaitReaderTime 7.483s | Percentage 0.00%
2022-11-30 13:41:56.786 [job-0] INFO StandAloneJobContainerCommunicator - Total 59904 records, 24085633 bytes | Speed 0B/s, 0 records/s | Error 0 records, 0 bytes | All Task WaitWriterTime 239.642s | All Task WaitReaderTime 7.483s | Percentage 0.00%
2022-11-30 13:42:06.787 [job-0] INFO StandAloneJobContainerCommunicator - Total 61952 records, 24909129 bytes | Speed 80.42KB/s, 204 records/s | Error 0 records, 0 bytes | All Task WaitWriterTime 252.721s | All Task WaitReaderTime 7.705s | Percentage 0.00%
2022-11-30 13:42:16.791 [job-0] INFO StandAloneJobContainerCommunicator - Total 64000 records, 25732613 bytes | Speed 80.42KB/s, 204 records/s | Error 0 records, 0 bytes | All Task WaitWriterTime 263.557s | All Task WaitReaderTime 7.980s | Percentage 0.00%
2022-11-30 13:42:26.797 [job-0] INFO StandAloneJobContainerCommunicator - Total 66048 records, 26555768 bytes | Speed 80.39KB/s, 204 records/s | Error 0 records, 0 bytes | All Task WaitWriterTime 275.084s | All Task WaitReaderTime 8.270s | Percentage 0.00%
2022-11-30 13:42:26.804 [job-0] INFO VMInfo -
[delta cpu info] =>
curDeltaCpu | averageCpu | maxDeltaCpu | minDeltaCpu
-1.00% | -1.00% | -1.00% | -1.00%

[delta memory info] =>
NAME | used_size | used_percent | max_used_size | max_percent
CodeHeap 'profiled nmethods' | 5.62MB | 82.55% | 5.62MB | 82.55%
G1 Old Gen | 312.95MB | 72.44% | 312.95MB | 72.44%
G1 Survivor Space | 52.00MB | 100.00% | 52.00MB | 100.00%
CodeHeap 'non-profiled nmethods' | 1.95MB | 79.90% | 1.95MB | 79.90%
Compressed Class Space | 2.78MB | 88.84% | 2.78MB | 88.84%
Metaspace | 27.72MB | 96.84% | 27.72MB | 96.84%
G1 Eden Space | 8.00MB | 1.48% | 8.00MB | 1.48%
CodeHeap 'non-nmethods' | 1.18MB | 48.32% | 1.18MB | 48.32%

[delta gc info] =>
NAME | curDeltaGCCount | totalGCCount | maxDeltaGCCount | minDeltaGCCount | curDeltaGCTime | totalGCTime | maxDeltaGCTime | minDeltaGCTime
G1 Young Generation | 117 | 117 | 117 | 117 | 1.833s | 1.833s | 1.833s | 1.833s
G1 Old Generation | 0 | 0 | 0 | 0 | 0.000s | 0.000s | 0.000s | 0.000s

2022-11-30 13:42:36.807 [job-0] INFO StandAloneJobContainerCommunicator - Total 68096 records, 27377309 bytes | Speed 80.23KB/s, 204 records/s | Error 0 records, 0 bytes | All Task WaitWriterTime 283.485s | All Task WaitReaderTime 8.444s | Percentage 0.00%
2022-11-30 13:42:46.808 [job-0] INFO StandAloneJobContainerCommunicator - Total 68096 records, 27377309 bytes | Speed 0B/s, 0 records/s | Error 0 records, 0 bytes | All Task WaitWriterTime 283.485s | All Task WaitReaderTime 8.444s | Percentage 0.00%
2022-11-30 13:42:56.813 [job-0] INFO StandAloneJobContainerCommunicator - Total 68096 records, 27377309 bytes | Speed 0B/s, 0 records/s | Error 0 records, 0 bytes | All Task WaitWriterTime 283.485s | All Task WaitReaderTime 8.444s | Percentage 0.00%
2022-11-30 13:43:06.814 [job-0] INFO StandAloneJobContainerCommunicator - Total 68096 records, 27377309 bytes | Speed 0B/s, 0 records/s | Error 0 records, 0 bytes | All Task WaitWriterTime 283.485s | All Task WaitReaderTime 8.444s | Percentage 0.00%
2022-11-30 13:43:16.815 [job-0] INFO StandAloneJobContainerCommunicator - Total 68096 records, 27377309 bytes | Speed 0B/s, 0 records/s | Error 0 records, 0 bytes | All Task WaitWriterTime 283.485s | All Task WaitReaderTime 8.444s | Percentage 0.00%
2022-11-30 13:43:26.816 [job-0] INFO StandAloneJobContainerCommunicator - Total 68096 records, 27377309 bytes | Speed 0B/s, 0 records/s | Error 0 records, 0 bytes | All Task WaitWriterTime 283.485s | All Task WaitReaderTime 8.444s | Percentage 0.00%
2022-11-30 13:43:36.817 [job-0] INFO StandAloneJobContainerCommunicator - Total 68096 records, 27377309 bytes | Speed 0B/s, 0 records/s | Error 0 records, 0 bytes | All Task WaitWriterTime 283.485s | All Task WaitReaderTime 8.444s | Percentage 0.00%
2022-11-30 13:43:46.818 [job-0] INFO StandAloneJobContainerCommunicator - Total 68096 records, 27377309 bytes | Speed 0B/s, 0 records/s | Error 0 records, 0 bytes | All Task WaitWriterTime 283.485s | All Task WaitReaderTime 8.444s | Percentage 0.00%
2022-11-30 13:43:56.821 [job-0] INFO StandAloneJobContainerCommunicator - Total 68096 records, 27377309 bytes | Speed 0B/s, 0 records/s | Error 0 records, 0 bytes | All Task WaitWriterTime 283.485s | All Task WaitReaderTime 8.444s | Percentage 0.00%
2022-11-30 13:44:06.824 [job-0] INFO StandAloneJobContainerCommunicator - Total 68096 records, 27377309 bytes | Speed 0B/s, 0 records/s | Error 0 records, 0 bytes | All Task WaitWriterTime 283.485s | All Task WaitReaderTime 8.444s | Percentage 0.00%
2022-11-30 13:44:16.826 [job-0] INFO StandAloneJobContainerCommunicator - Total 68096 records, 27377309 bytes | Speed 0B/s, 0 records/s | Error 0 records, 0 bytes | All Task WaitWriterTime 283.485s | All Task WaitReaderTime 8.444s | Percentage 0.00%
2022-11-30 13:44:26.831 [job-0] INFO StandAloneJobContainerCommunicator - Total 68096 records, 27377309 bytes | Speed 0B/s, 0 records/s | Error 0 records, 0 bytes | All Task WaitWriterTime 283.485s | All Task WaitReaderTime 8.444s | Percentage 0.00%
2022-11-30 13:44:27.295 [0-0-0-writer] WARN CommonRdbmsWriter$Task - 回滚此次写入, 采用每次写入一行方式提交. 因为:Communications link failure

Contributor guide

No contributing guide indexed for this repository

Research direction

The report names no source files, entry points, or tests. Start by reproducing the SQL Server-to-MySQL transfer while collecting the complete job configuration and database logs. Done means explaining and correcting the declining transfer speed and MySQL shutdown, with evidence that the transfer remains stable.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, mysql, sql
Domain
data-engineering, databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.