alibaba / alibaba/DataX

在同步时遇到了一个报错没有找到原因tdenginewritter. 同步场景为td-td

Open
#1,576 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
17.4k
Forks
5.7k
PR merge metrics
No merged PRs in 30d

Description

com.alibaba.datax.common.exception.DataXException: Code:[TDengineWriter-02], Description:[runtime exception]. - cannot find col: ts in columns: [ts, i_a, i_b, i_c, i_sum, elc, u_a, u_b, u_c, power, corp_id, equipid, line_id]
at com.alibaba.datax.common.exception.DataXException.asDataXException(DataXException.java:30) ~[datax-common-0.0.1-SNAPSHOT.jar:na]
at com.alibaba.datax.plugin.writer.tdenginewriter.DefaultDataHandler.indexOf(DefaultDataHandler.java:552) [tdenginewriter-0.0.1-SNAPSHOT.jar:na]
at com.alibaba.datax.plugin.writer.tdenginewriter.DefaultDataHandler.writeBatchToSupTableBySchemaless(DefaultDataHandler.java:317) [tdenginewriter-0.0.1-SNAPSHOT.jar:na]
at com.alibaba.datax.plugin.writer.tdenginewriter.DefaultDataHandler.writeBatch(DefaultDataHandler.java:158) [tdenginewriter-0.0.1-SNAPSHOT.jar:na]
at com.alibaba.datax.plugin.writer.tdenginewriter.DefaultDataHandler.writeEachRow(DefaultDataHandler.java:129) [tdenginewriter-0.0.1-SNAPSHOT.jar:na]
at com.alibaba.datax.plugin.writer.tdenginewriter.DefaultDataHandler.handle(DefaultDataHandler.java:96) [tdenginewriter-0.0.1-SNAPSHOT.jar:na]
at com.alibaba.datax.plugin.writer.tdenginewriter.TDengineWriter$Task.startWrite(TDengineWriter.java:109) [tdenginewriter-0.0.1-SNAPSHOT.jar:na]
at com.alibaba.datax.core.taskgroup.runner.WriterRunner.run(WriterRunner.java:56) [datax-core-0.0.1-SNAPSHOT.jar:na]
at java.lang.Thread.run(Thread.java:750) [na:1.8.0_345]
脚本如下:
{
"content":[
{
"reader":{
"name":"tdenginereader",
"parameter":{
"beginDateTime":"2022-10-25 17:47:31",
"column":[
"ts",
"i_a",
"i_b",
"i_c",
"i_sum",
"elc",
"u_a",
"u_b",
"u_c",
"power",
"corpid",
"equipid",
"lineid"
],
"connection":[
{
"jdbcUrl":[
"jdbc:TAOS-RS://xxxxxxx:6041/gkjk?timestampFormat=TIMESTAMP"
],
"table":[
"equip_min"
]
}
],
"endDateTime":"2022-10-25 17:50:31",
"password":"",
"splitInterval":"5m",
"username":"hdec"
}
},
"writer":{
"name":"tdenginewriter",
"parameter":{
"batchSize":1000,
"column":[
"ts",
"i_a",
"i_b",
"i_c",
"i_sum",
"elc",
"u_a",
"u_b",
"u_c",
"power",
"corp_id",
"equipid",
"line_id"
],
"connection":[
{
"jdbcUrl":"jdbc:TAOS://xxxxxxx:6030/test?timestampFormat=TIMESTAMP",
"table":[
"abandoned_water_elc_data"
]
}
],
"ignoreTagsUnmatched":true,
"password":"",
"username":"root"
}
}
}
],
"setting":{
"speed":{
"channel":1
}
}
}

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with the stack trace in plugin/writer/tdenginewriter/DefaultDataHandler.java, especially indexOf at line 552 and writeBatchToSupTableBySchemaless at line 317. Compare the reader and writer column lists in the supplied DataX configuration, then reproduce the td-to-td sync; done means the reported column lookup failure is explained and the configuration or handling has a verified resolution.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
databases
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.