DTStack / DTStack/chunjun

[Question][chunjun-core] binlog-mysql数据转换(transformer)错误

Open
#1,431 0 comments 0 reactions 0 assignees View on GitHub
question
Dominant language
Java
Stars
4.1k
Forks
1.7k
PR merge metrics
No merged PRs in 30d

Description

### Search before asking

- [x] I had searched in the [issues](https://github.com/DTStack/chunjun/issues) and found no similar question.

- [X] I had googled my question but i didn't get any help.

- [X] I had read the documentation: [ChunJun doc](https://dtstack.github.io/chunjun) but it didn't help me.

### Description

binlog-mysql,请问是哪里出错了,会报一个flink的错。
```console
Exception in thread "main" org.apache.flink.table.api.ValidationException: Too many fields referenced from an atomic type.
at org.apache.flink.table.typeutils.FieldInfoUtils.extractFieldInfoFromAtomicType(FieldInfoUtils.java:471)
at org.apache.flink.table.typeutils.FieldInfoUtils.extractFieldInformation(FieldInfoUtils.java:294)
at org.apache.flink.table.typeutils.FieldInfoUtils.getFieldsInfo(FieldInfoUtils.java:258)
at org.apache.flink.table.api.bridge.java.internal.StreamTableEnvironmentImpl.lambda$asQueryOperation$0(StreamTableEnvironmentImpl.java:388)
at java.util.Optional.map(Optional.java:215)
at org.apache.flink.table.api.bridge.java.internal.StreamTableEnvironmentImpl.asQueryOperation(StreamTableEnvironmentImpl.java:385)
at org.apache.flink.table.api.bridge.java.internal.StreamTableEnvironmentImpl.fromDataStream(StreamTableEnvironmentImpl.java:232)
at com.dtstack.chunjun.Main.syncStreamToTable(Main.java:250)
at com.dtstack.chunjun.Main.exeSyncJob(Main.java:212)
at com.dtstack.chunjun.Main.main(Main.java:122)
at com.dtstack.chunjun.local.test.LocalTest.main(LocalTest.java:173)
```
这个错是用多个字段就会报错,但是我用单个字段chunjun会报错说找不到sourceTable.xxx字段,请问是哪里出了问题?

下面是json
```json
{
"job": {
"content": [
{
"nameMapping": {
"schemaMappings": {
"jiahedb": "dqq_db"
},
"tableMappings": {
"jiahedb": {
"testest": "jiahe_2"
}
}
},
"reader": {
"name": "binlogreader",
"table": {
"tableName": "sourceTable"
},
"fieldList": [
{
"name": "jid",
"type": "int"
},
{
"name": "jname",
"type": "varchar"
},
{
"name": "jdouble",
"type": "double"
}
],
"fieldNameList": [
"jid",
"jname",
"jdouble"
],
"parameter": {
"fieldMapping": {
"jid": "ijd",
"jname": "namej",
"jdouble": "before_jdouble"
},
"column": [
{
"name": "jid",
"type": "int"
},
{
"name": "jname",
"type": "varchar"
},
{
"name": "jdouble",
"type": "double"
}
],
"jdbcUrl": "jdbc:mysql://",
"host": "",
"port":,
"cat": "insert,update,delete",
"readPosition": "current",
"password": "",
"username": "root"
}
},
"writer": {
"name": "mysqlwriter",
"table": {
"tableName": "sinkTable"
},
"parameter": {
"connection": [
{
"jdbcUrl": "jdbc:mysql://",
"table": [
"*"
]
}
],
"column": [
{
"name": "after_jid",
"type": "int"
},
{
"name": "after_jname",
"type": "varchar"
},
{
"name": "after_jdouble",
"type": "double"
}
],
"password": "",
"writeMode": "insert",
"username": ""
}
},
"transformer": {
"transformSql": "select jid,jname,jdouble from sourceTable where jid = 456"
}
}
],
"setting": {
"errorLimit": {
"record": 0
},
"speed": {
"bytes": 0,
"channel": 1
}
}
}
}
```

### Code of Conduct

- [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.