sqlserver to databend报错
- Dominant language
- Java
- Stars
- 17.4k
- Forks
- 5.7k
- PR merge metrics
- No merged PRs in 30d
Description
[root@mysql-80-24 DataX]# python ./target/datax/datax/bin/datax.py ./sqlserver_xx.json
DataX (DATAX-OPENSOURCE-3.0), From Alibaba !
Copyright (C) 2010-2017, Alibaba Group. All Rights Reserved.
2023-05-21 07:38:37.762 [main] INFO MessageSource - JVM TimeZone: GMT+08:00, Locale: zh_CN
2023-05-21 07:38:37.765 [main] INFO MessageSource - use Locale: zh_CN timeZone: sun.util.calendar.ZoneInfo[id="GMT+08:00",offset=28800000,dstSavings=0,useDaylight=false,transitions=0,lastRule=null]
2023-05-21 07:38:37.798 [main] INFO VMInfo - VMInfo# operatingSystem class => sun.management.OperatingSystemImpl
2023-05-21 07:38:37.804 [main] INFO Engine - the machine info =>
osInfo: Red Hat, Inc. 1.8 25.372-b07
jvmInfo: Linux amd64 3.10.0-1160.24.1.el7.x86_64
cpu num: 16
totalPhysicalMemory: -0.00G
freePhysicalMemory: -0.00G
maxFileDescriptorCount: -1
currentOpenFileDescriptorCount: -1
GC Names [PS MarkSweep, PS Scavenge]
MEMORY_NAME | allocation_size | init_size
PS Eden Space | 256.00MB | 256.00MB
Code Cache | 240.00MB | 2.44MB
Compressed Class Space | 1,024.00MB | 0.00MB
PS Survivor Space | 42.50MB | 42.50MB
PS Old Gen | 683.00MB | 683.00MB
Metaspace | -0.00MB | 0.00MB
2023-05-21 07:38:37.818 [main] INFO Engine -
{
"content":[
{
"reader":{
"name":"sqlserverreader",
"parameter":{
"column":[
"a"
],
"connection":[
{
"jdbcUrl":[
"jdbc:sqlserver://xxx:xx;DatabaseName=test"
],
"table":[
"s"
]
}
],
"password":"******",
"username":"test_user"
}
},
"writer":{
"name":"databendwriter",
"parameter":{
"column":[
"a"
],
"connection":[
{
"jdbcUrl":"jdbc:databend://xx:8000/migrate_db",
"table":"s"
}
],
"maxBatchRows":65536,
"maxBatchSize":134217728,
"password":"*********",
"postSql":[
],
"preSql":[
],
"username":"xx"
}
}
}
],
"setting":{
"speed":{
"channel":""
}
}
}
2023-05-21 07:38:37.838 [main] INFO PerfTrace - PerfTrace traceId=job_-1, isEnable=false
2023-05-21 07:38:37.839 [main] INFO JobContainer - DataX jobContainer starts job.
2023-05-21 07:38:37.840 [main] INFO JobContainer - Set jobId = 0
2023-05-21 07:38:38.368 [job-0] INFO OriginalConfPretreatmentUtil - Available jdbcUrl:jdbc:sqlserver://172.22.22.67:52016;DatabaseName=test.
2023-05-21 07:38:38.463 [job-0] INFO OriginalConfPretreatmentUtil - table:[s] has columns:[a].
2023-05-21 07:38:38.484 [job-0] ERROR JobContainer - Exception when job run
java.lang.ClassCastException: java.lang.String cannot be cast to java.util.List
at com.alibaba.datax.common.util.Configuration.getList(Configuration.java:435) ~[datax-common-0.0.1-SNAPSHOT.jar:na]
at com.alibaba.datax.plugin.rdbms.writer.util.OriginalConfPretreatmentUtil.simplifyConf(OriginalConfPretreatmentUtil.java:70) ~[plugin-rdbms-util-0.0.1-SNAPSHOT.jar:na]
at com.alibaba.datax.plugin.rdbms.writer.util.OriginalConfPretreatmentUtil.doPretreatment(OriginalConfPretreatmentUtil.java:34) ~[plugin-rdbms-util-0.0.1-SNAPSHOT.jar:na]
at com.alibaba.datax.plugin.rdbms.writer.CommonRdbmsWriter$Job.init(CommonRdbmsWriter.java:41) ~[plugin-rdbms-util-0.0.1-SNAPSHOT.jar:na]
at com.alibaba.datax.plugin.writer.databendwriter.DatabendWriter$Job.init(DatabendWriter.java:36) ~[databendwriter-0.0.1-SNAPSHOT.jar:na]
at com.alibaba.datax.core.job.JobContainer.initJobWriter(JobContainer.java:704) ~[datax-core-0.0.1-SNAPSHOT.jar:na]
at com.alibaba.datax.core.job.JobContainer.init(JobContainer.java:304) ~[datax-core-0.0.1-SNAPSHOT.jar:na]
at com.alibaba.datax.core.job.JobContainer.start(JobContainer.java:113) ~[datax-core-0.0.1-SNAPSHOT.jar:na]
at com.alibaba.datax.core.Engine.start(Engine.java:86) [datax-core-0.0.1-SNAPSHOT.jar:na]
at com.alibaba.datax.core.Engine.entry(Engine.java:168) [datax-core-0.0.1-SNAPSHOT.jar:na]
at com.alibaba.datax.core.Engine.main(Engine.java:201) [datax-core-0.0.1-SNAPSHOT.jar:na]
2023-05-21 07:38:38.489 [job-0] INFO StandAloneJobContainerCommunicator - Total 0 records, 0 bytes | Speed 0B/s, 0 records/s | Error 0 records, 0 bytes | All Task WaitWriterTime 0.000s | All Task WaitReaderTime 0.000s | Percentage 0.00%
2023-05-21 07:38:38.495 [job-0] ERROR Engine -
经DataX智能分析,该任务最可能的错误原因是:
com.alibaba.datax.common.exception.DataXException: Code:[Framework-02], Description:[DataX引擎运行过程出错,具体原因请参看DataX运行结束时的错误诊断信息 .]. - java.lang.ClassCastException: java.lang.String cannot be cast to java.util.List
at com.alibaba.datax.common.util.Configuration.getList(Configuration.java:435)
at com.alibaba.datax.plugin.rdbms.writer.util.OriginalConfPretreatmentUtil.simplifyConf(OriginalConfPretreatmentUtil.java:70)
at com.alibaba.datax.plugin.rdbms.writer.util.OriginalConfPretreatmentUtil.doPretreatment(OriginalConfPretreatmentUtil.java:34)
at com.alibaba.datax.plugin.rdbms.writer.CommonRdbmsWriter$Job.init(CommonRdbmsWriter.java:41)
at com.alibaba.datax.plugin.writer.databendwriter.DatabendWriter$Job.init(DatabendWriter.java:36)
at com.alibaba.datax.core.job.JobContainer.initJobWriter(JobContainer.java:704)
at com.alibaba.datax.core.job.JobContainer.init(JobContainer.java:304)
at com.alibaba.datax.core.job.JobContainer.start(JobContainer.java:113)
at com.alibaba.datax.core.Engine.start(Engine.java:86)
at com.alibaba.datax.core.Engine.entry(Engine.java:168)
at com.alibaba.datax.core.Engine.main(Engine.java:201)
- java.lang.ClassCastException: java.lang.String cannot be cast to java.util.List
at com.alibaba.datax.common.util.Configuration.getList(Configuration.java:435)
at com.alibaba.datax.plugin.rdbms.writer.util.OriginalConfPretreatmentUtil.simplifyConf(OriginalConfPretreatmentUtil.java:70)
at com.alibaba.datax.plugin.rdbms.writer.util.OriginalConfPretreatmentUtil.doPretreatment(OriginalConfPretreatmentUtil.java:34)
at com.alibaba.datax.plugin.rdbms.writer.CommonRdbmsWriter$Job.init(CommonRdbmsWriter.java:41)
at com.alibaba.datax.plugin.writer.databendwriter.DatabendWriter$Job.init(DatabendWriter.java:36)
at com.alibaba.datax.core.job.JobContainer.initJobWriter(JobContainer.java:704)
at com.alibaba.datax.core.job.JobContainer.init(JobContainer.java:304)
at com.alibaba.datax.core.job.JobContainer.start(JobContainer.java:113)
at com.alibaba.datax.core.Engine.start(Engine.java:86)
at com.alibaba.datax.core.Engine.entry(Engine.java:168)
at com.alibaba.datax.core.Engine.main(Engine.java:201)
at com.alibaba.datax.common.exception.DataXException.asDataXException(DataXException.java:48)
at com.alibaba.datax.core.job.JobContainer.start(JobContainer.java:162)
at com.alibaba.datax.core.Engine.start(Engine.java:86)
at com.alibaba.datax.core.Engine.entry(Engine.java:168)
at com.alibaba.datax.core.Engine.main(Engine.java:201)
Caused by: java.lang.ClassCastException: java.lang.String cannot be cast to java.util.List
at com.alibaba.datax.common.util.Configuration.getList(Configuration.java:435)
at com.alibaba.datax.plugin.rdbms.writer.util.OriginalConfPretreatmentUtil.simplifyConf(OriginalConfPretreatmentUtil.java:70)
at com.alibaba.datax.plugin.rdbms.writer.util.OriginalConfPretreatmentUtil.doPretreatment(OriginalConfPretreatmentUtil.java:34)
at com.alibaba.datax.plugin.rdbms.writer.CommonRdbmsWriter$Job.init(CommonRdbmsWriter.java:41)
at com.alibaba.datax.plugin.writer.databendwriter.DatabendWriter$Job.init(DatabendWriter.java:36)
at com.alibaba.datax.core.job.JobContainer.initJobWriter(JobContainer.java:704)
at com.alibaba.datax.core.job.JobContainer.init(JobContainer.java:304)
at com.alibaba.datax.core.job.JobContainer.start(JobContainer.java:113)
... 3 more
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with the reported configuration and the stack trace at Configuration.java:435 and OriginalConfPretreatmentUtil.java:70, then trace initialization through DatabendWriter.java:36. Run the shown DataX job with the SQL Server reader and Databend writer configuration; done means the job no longer fails during writer initialization with the reported ClassCastException.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- databases
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100