DTStack / DTStack/chunjun

yarn模式运行flink sql的任务,当字段类型为timestamp时候报错

Open
#593 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
4.1k
Forks
1.7k
PR merge metrics
No merged PRs in 30d

Description

使用flinkx 1.12_release提交一个flink sql的任务,source是mysql,里面有个字段的类型是"type" : "TIMESTAMP(6)",当我使用yarn-per-job运行任务的时候,出现如下的错误:
ERROR com.dtstack.flinkx.connector.jdbc.converter.JdbcRowConverter [] - **value [2022-01-12T13:32:36] convent failed**

自己调查了一下,flinkx的代码中出错的地方在JdbcRowConverter.java的return TimestampData.fromTimestamp((Timestamp) val); 发现val的类型是java.time.LocalDateTime,不是java.sql.Timestamp。 当我使用local模式运行的时候,却没有出现类似的错误,val的类型是java.sql.Timestamp。请问这是什么原因导致的呢?

重要的日志:
[JdbcConf]:
{
"semantic" : "at-least-once",
"errorRecord" : 0,
"checkFormat" : true,
"parallelism" : 1,
"pollingInterval" : 5000,
"increment" : true,
"flushIntervalMills" : 10000,
"polling" : false,
"querySql" : "SELECT `id`,`status`, `create_time`, `create_user` FROM `sys_role_permission` WHERE `id` > 36096 ORDER BY `id` ASC",
"mode" : "INSERT",
"password" : "******",
"startLocation" : "36096",
"increColumn" : "id",
"restoreColumn" : "id",
"restoreColumnIndex" : 0,
"connection" : [ {
"table" : [ "sys_role_permission" ],
"jdbcUrl" : [ "jdbc:mysql://xxxxx:3306/xxxx?useUnicode=true&characterEncoding=UTF-8&serverTimezone=GMT&useSSL=false" ]
} ],
"splitPk" : "id",
"table" : "sys_role_permission",
"queryTimeOut" : 0,
"restoreColumnType" : "BIGINT",
"fetchSize" : -2147483648,
"useMaxFunc" : false,
"column" : [ {
"name" : "id",
"type" : "BIGINT",
"index" : 0,
"notNull" : false,
"part" : false
}, {
"name" : "status",
"type" : "TINYINT",
"index" : 5,
"notNull" : false,
"part" : false
}, {
"name" : "create_time",
"type" : "TIMESTAMP(6)",
"index" : 6,
"notNull" : false,
"part" : false
}, {
"name" : "create_user",
"type" : "STRING",
"index" : 7,
"notNull" : false,
"part" : false
} ],
"errorPercentage" : -1,
"fieldNameList" : [ ],
"withNoLock" : false,
"increColumnIndex" : 0,
"allReplace" : false,
"splitStrategy" : "range",
"jdbcUrl" : "jdbc:mysql://xxxx:3306/xxxx?useUnicode=true&characterEncoding=UTF-8&serverTimezone=Asia/Shanghai&useSSL=false",
"batchSize" : 1,
"speedBytes" : 0,
"metricPluginName" : "prometheus",
"increColumnType" : "BIGINT",
"username" : "xxxx"
}

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.