DTStack / DTStack/chunjun

chunjun执行mysql-mysql insert模式下问题

Open
#1,175 2 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
Java
Stars
4.1k
Forks
1.7k
PR merge metrics
No merged PRs in 30d

Description

### Search before asking

- [X] I had searched in the [issues](https://github.com/DTStack/chunjun/issues) and found no similar issues.

### What happened

问题一:insert 模式下 主键重复 任务直接终止
问题二:insert模式下 配置脏数据存储 也没有用 任务终止 ,张数据表也无数据

### What you expected to happen

1、insert模式下 mysql-mysql 主键重复直接统计到 统计指标中
2、脏数据入库 且不会终止任务

### How to reproduce

1、执行jSON
{
"job": {
"content": [
{
"reader": {
"parameter": {
"password": "123456",
"dataSourceId": 8,
"column": [
{
"precision": 4,
"name": "id",
"columnDisplaySize": 4,
"type": "INT"
},
{
"precision": 255,
"name": "name",
"columnDisplaySize": 255,
"type": "VARCHAR"
},
{
"precision": 4,
"name": "age",
"columnDisplaySize": 4,
"type": "TINYINT"
},
{
"precision": 255,
"name": "phone",
"columnDisplaySize": 255,
"type": "VARCHAR"
}
],
"connection": [
{
"jdbcUrl": [
"jdbc:mysql://172.18.8.77:3306/zk_test"
],
"table": [
"one"
]
}
],
"increColumn":"id",
"splitPk": "id",
"username": "root"
},
"name": "mysqlreader"
},
"writer": {
"parameter": {
"password": "123456",
"dataSourceId": 8,
"column": [
{
"precision": 4,
"name": "id",
"columnDisplaySize": 4,
"type": "INT"
},
{
"precision": 255,
"name": "name",
"columnDisplaySize": 255,
"type": "VARCHAR"
},
{
"precision": 4,
"name": "age",
"columnDisplaySize": 4,
"type": "TINYINT"
},
{
"precision": 255,
"name": "phone",
"columnDisplaySize": 255,
"type": "VARCHAR"
}
],
"connection": [
{
"jdbcUrl": "jdbc:mysql://172.18.8.77:3306/zk_test",
"table": [
"one_copy1"
]
}
],
"writeMode": "insert",
"username": "root"
},
"name": "mysqlwriter"
}
}
],
"setting": {
"log": {
"isLogger": false
},
"errorLimit": {},
"speed": {
"bytes": 0,
"channel": 1
}
}
}
}

2、脏数据配置
{\"start-chunjun.dirty-data.output-type\":\"jdbc\",\"start-chunjun.dirty-data.max-rows\":\"1000\",\"start-chunjun.dirty-data.max-collect-failed-rows\":\"100\",\"start-chunjun.dirty-data.jdbc.url\":\"jdbc:mysql://172.18.8.77:3306/zk_test\",\"start-chunjun.dirty-data.jdbc.username\":\"root\",\"start-chunjun.dirty-data.jdbc.password\":\"123456\",\"start-chunjun.dirty-data.jdbc.table\":\"chunjun_dirty_data\",\"start-chunjun.dirty-data.jdbc.batch-size\":\"10\",\"start-chunjun.dirty-data.log.print-interval\":\"10\"}

### Anything else

_No response_

### Version

master

### Are you willing to submit PR?

- [X] Yes I am willing to submit a PR!

### Code of Conduct

- [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.