alibaba / alibaba/DataX

datax写入mysql速度慢

Open
#944 3 comments 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
17.4k
Forks
5.7k
PR merge metrics
No merged PRs in 30d

Description

datax从clickhouse写入mysql耗时,大概50万左右速度开始断崖下降,越往后越慢,时间都耗在写入了
![image](https://user-images.githubusercontent.com/29367667/104812315-53ac1280-583c-11eb-9afd-7b0669288dff.png)
![image](https://user-images.githubusercontent.com/29367667/104812370-a4237000-583c-11eb-8dcf-d93a861c14ab.png)
![image](https://user-images.githubusercontent.com/29367667/104812383-bb625d80-583c-11eb-90a5-72f9f88200bd.png)

Contributor guide

No contributing guide indexed for this repository

Research direction

The issue names only the ClickHouse-to-MySQL transfer and provides screenshots; no files or tests are identified. Start by reproducing the slowdown around 500,000 rows and collecting the job configuration and environment details. Done means identifying and validating a concrete cause or fix for the progressive MySQL write slowdown.

Written by the indexing model from the issue text.

Assessment

Tech stack
clickhouse, java, mysql
Domain
data-engineering, databases, performance
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.