使用mysqlreader读取ADS数据库的数据,发生字段获取不正确导致类型转换报错的问题。
- Dominant language
- Java
- Stars
- 17.4k
- Forks
- 5.7k
- PR merge metrics
- No merged PRs in 30d
Description
2022-11-10 18:03:50.503 [job-0] ERROR JobContainer - 运行scheduler 模式[standalone]出错.
2022-11-10 18:03:50.504 [job-0] ERROR JobContainer - Exception when job run
com.alibaba.datax.common.exception.DataXException: Code:[DBUtilErrorCode-07], Description:[读取数据库数据失败. 请检查您的配置的 column/table/where/querySql或者向 DBA 寻求帮助.]. - 执行的SQL为: (select * from ...) 具体错误信息为:com.alibaba.datax.common.exception.DataXException: Code:[Common-01], Describe:[同步数据出现业务脏数据情况,数据类型转换错误 .] - String[ 211 ���������321113���������������������41126100000050534(2013)������������������0148���0������������������������������������������������� ���������148�����2150���������]不能转为Long .
at com.alibaba.datax.common.exception.DataXException.asDataXException(DataXException.java:26) ~[datax-common-0.0.1-SNAPSHOT.jar:na]
at com.alibaba.datax.plugin.rdbms.util.RdbmsException.asQueryException(RdbmsException.java:81) ~[na:na]
at com.alibaba.datax.plugin.rdbms.reader.CommonRdbmsReader$Task.startRead(CommonRdbmsReader.java:220) ~[na:na]
at com.alibaba.datax.plugin.reader.mysqlreader.MysqlReader$Task.startRead(MysqlReader.java:81) ~[na:na]
at com.alibaba.datax.core.taskgroup.runner.ReaderRunner.run(ReaderRunner.java:57) ~[datax-core-0.0.1-SNAPSHOT.jar:na]
at java.lang.Thread.run(Thread.java:748) ~[na:1.8.0_161]
2022-11-10 18:03:50.504 [job-0] INFO StandAloneJobContainerCommunicator - Total 6307457 records, 3419464546 bytes | Speed 3.18GB/s, 6307457 records/s | Error 1 records, 0 bytes | All Task WaitWriterTime 6,397.061s | All Task WaitReaderTime 0.636s | Percentage 0.00%
2022-11-10 18:03:50.505 [job-0] ERROR Engine -
经DataX智能分析,该任务最可能的错误原因是:
com.alibaba.datax.common.exception.DataXException: Code:[DBUtilErrorCode-07], Description:[读取数据库数据失败. 请检查您的配置的 column/table/where/querySql或者向 DBA 寻求帮助.]. - 执行的SQL为: (省略) 具体错误信息为:com.alibaba.datax.common.exception.DataXException: Code:[Common-01], Describe:[同步数据出现业务脏数据情况,数据类型转换错误 .] - String[ 211 ���������321113���������������������41126100000050534(2013)������������������0148���0������������������������������������������������� ���������148�����2150���������]不能转为Long .
at com.alibaba.datax.common.exception.DataXException.asDataXException(DataXException.java:26)
at com.alibaba.datax.plugin.rdbms.util.RdbmsException.asQueryException(RdbmsException.java:81)
at com.alibaba.datax.plugin.rdbms.reader.CommonRdbmsReader$Task.startRead(CommonRdbmsReader.java:220)
at com.alibaba.datax.plugin.reader.mysqlreader.MysqlReader$Task.startRead(MysqlReader.java:81)
at com.alibaba.datax.core.taskgroup.runner.ReaderRunner.run(ReaderRunner.java:57)
at java.lang.Thread.run(Thread.java:748)
[INFO] 2022-11-10 18:03:51.489 TaskLogLogger-class org.apache.dolphinscheduler.plugin.task.datax.DataxTask:[60] - FINALIZE_SESSION
正常应该获取这条数据的第一个字段,然后转成datax的Long类型,但是获取第一个字段的时候,获取了这条数据多个字段和分隔符组成一个字段,导致数据类型转换失败。
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with the stack-trace entry points in plugin/rdbms/src/main/java/com/alibaba/datax/plugin/rdbms/reader/CommonRdbmsReader.java and plugin/reader/mysqlreader/src/main/java/com/alibaba/datax/plugin/reader/mysqlreader/MysqlReader.java. Reproduce the mysqlreader failure against the reported ADS database and inspect how the first field is obtained before conversion. Done means the first field is read separately and the reported value converts to DataX Long without combining multiple fields.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java, mysql
- Domain
- database
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100