alibaba / alibaba/DataX

datax同步mysql到doris,数据量多了一千多万

Open
#1,557 7 comments 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
17.4k
Forks
5.7k
PR merge metrics
No merged PRs in 30d

Description

![image](https://user-images.githubusercontent.com/100275749/196924372-5e7e4e75-f530-4a6c-846a-83a7da6705b1.png)
原表只有2600w多数据,抽取结束显示这么多。另外两个表倒是正常。
json配置没什么特别的,就是用了原表的id作为splitPk,配置了table,reader的column为*,数据库写在了jdbcUrl中。channel配置了4个,限速为-1。

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reproducing the MySQL-to-Doris sync with the reported JSON settings: id as splitPk, table and reader column set to *, four channels, and no rate limit. Compare source and destination row counts and inspect the configuration and logs to identify why roughly ten million extra rows are written; done means the counts match.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, mysql
Domain
data-engineering, databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.