alibaba / alibaba/DataX

[bug]datax把mysql数据迁移到ob,使用"preSql": ["truncate table XXX"],无数据

Open
#2,258 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
17.4k
Forks
5.7k
PR merge metrics
No merged PRs in 30d

Description

{
"job": {
"setting": {
"speed": {
"channel": 4
},
"errorLimit": {
"record": 0,
"percentage": 0.1
}
},
"content": [
{
"reader": {
"name": "mysqlreader",
"parameter": {
"username": "root",
"password": "XXXXX",
"column": [
"*"
],
"where": "dt='${bizdate}'",
"connection": [
{
"table": ["XXXXX"],
"jdbcUrl": ["jdbc:mysql://XXXX:3306/za_ods?useUnicode=true&characterEncoding=utf8&serverTimezone=Asia/Shanghai"]
}
]
}
},
"writer": {
"name": "oceanbasev10writer",
"parameter": {
"obWriteMode": "insert",
"column": [
"*"
],
"preSql": ["truncate table XXXXX"],
"connection": [
{
"jdbcUrl": "||_dsc_ob10_dsc_||obyyt:nldz||_dsc_ob10_dsc_||jdbc:oceanbase://XXXX:2883/dgped_bus_zaxq?useLocalSessionState=true&allowBatch=true&allowMultiQueries=true&rewriteBatchedStatements=true",
"table": ["XXXX"]
}
],
"username": "root",
"password": "XXXX",
"writerThreadCount": 10,
"batchSize": 1000,
"memstoreThreshold": "0.9"
}
}
}
]
}
}

加上"preSql": ["truncate table XXXXX"], 后,发现任务能运行成功,但是表里没有数据。去掉这个后,没问题。

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reproducing the DataX job with the oceanbasev10writer configuration, comparing runs with and without the preSql truncate statement. Inspect how the writer executes preSql before writing, and consider the issue complete when truncation and subsequent data insertion both succeed with rows present in the target table.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, mysql
Domain
databases
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.