chunjun执行mysql-mysql insert模式下问题
- Dominant language
- Java
- Stars
- 4.1k
- Forks
- 1.7k
- PR merge metrics
- No merged PRs in 30d
Description
### Search before asking
- [X] I had searched in the [issues](https://github.com/DTStack/chunjun/issues) and found no similar issues.
### What happened
问题一:insert 模式下 主键重复 任务直接终止
问题二:insert模式下 配置脏数据存储 也没有用 任务终止 ,张数据表也无数据
### What you expected to happen
1、insert模式下 mysql-mysql 主键重复直接统计到 统计指标中
2、脏数据入库 且不会终止任务
### How to reproduce
1、执行jSON
{
"job": {
"content": [
{
"reader": {
"parameter": {
"password": "123456",
"dataSourceId": 8,
"column": [
{
"precision": 4,
"name": "id",
"columnDisplaySize": 4,
"type": "INT"
},
{
"precision": 255,
"name": "name",
"columnDisplaySize": 255,
"type": "VARCHAR"
},
{
"precision": 4,
"name": "age",
"columnDisplaySize": 4,
"type": "TINYINT"
},
{
"precision": 255,
"name": "phone",
"columnDisplaySize": 255,
"type": "VARCHAR"
}
],
"connection": [
{
"jdbcUrl": [
"jdbc:mysql://172.18.8.77:3306/zk_test"
],
"table": [
"one"
]
}
],
"increColumn":"id",
"splitPk": "id",
"username": "root"
},
"name": "mysqlreader"
},
"writer": {
"parameter": {
"password": "123456",
"dataSourceId": 8,
"column": [
{
"precision": 4,
"name": "id",
"columnDisplaySize": 4,
"type": "INT"
},
{
"precision": 255,
"name": "name",
"columnDisplaySize": 255,
"type": "VARCHAR"
},
{
"precision": 4,
"name": "age",
"columnDisplaySize": 4,
"type": "TINYINT"
},
{
"precision": 255,
"name": "phone",
"columnDisplaySize": 255,
"type": "VARCHAR"
}
],
"connection": [
{
"jdbcUrl": "jdbc:mysql://172.18.8.77:3306/zk_test",
"table": [
"one_copy1"
]
}
],
"writeMode": "insert",
"username": "root"
},
"name": "mysqlwriter"
}
}
],
"setting": {
"log": {
"isLogger": false
},
"errorLimit": {},
"speed": {
"bytes": 0,
"channel": 1
}
}
}
}
2、脏数据配置
{\"start-chunjun.dirty-data.output-type\":\"jdbc\",\"start-chunjun.dirty-data.max-rows\":\"1000\",\"start-chunjun.dirty-data.max-collect-failed-rows\":\"100\",\"start-chunjun.dirty-data.jdbc.url\":\"jdbc:mysql://172.18.8.77:3306/zk_test\",\"start-chunjun.dirty-data.jdbc.username\":\"root\",\"start-chunjun.dirty-data.jdbc.password\":\"123456\",\"start-chunjun.dirty-data.jdbc.table\":\"chunjun_dirty_data\",\"start-chunjun.dirty-data.jdbc.batch-size\":\"10\",\"start-chunjun.dirty-data.log.print-interval\":\"10\"}
### Anything else
_No response_
### Version
master
### Are you willing to submit PR?
- [X] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
Contributor guide
Assessment
This issue has not been assessed yet.