alibaba / alibaba/DataX

迁移tdengine2.1.3.0版本到tgengine3.2.3.0报错java.sql.SQLException: ERROR (2318): Read timed out

Open
#2,214 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
17.4k
Forks
5.7k
PR merge metrics
No merged PRs in 30d

Description

迁移tdengine2.1.3.0版本-->3.2.3.0
使用datax工具,python bin/datax.py --jvm="-Xms4G -Xmx4G" script/tdengine2.xto3.x.json
script/tdengine2.xto3.x.json
{
"job": {
"content": [{
"reader": {
"name": "tdengine20reader",
"parameter": {
"username": "root",
"password": "taosdata",
"connection": [{
"querySql": [
"select tbname,* from db.sb_fhsj where sjsj >= '2024-01-01 00:00:00' and sjsj < '2024-02-01 00:00:00' limit 3788801,3891200"
],
"jdbcUrl": [
"jdbc:TAOS-RS://172.16.203.172:6041/db?timestampFormat=TIMESTAMP"
]
}]
}
},
"writer": {
"name": "tdengine30writer",
"parameter": {
"username": "root",
"password": "*",
"column": [
"tbname",
"sjsj",
"jssj",
"yxzt",
"sjzt",
"jdqzt",
"axdy",
"bxdy",
"cxdy",
"axdl",
"bxdl",
"cxdl",
"sydl",
"zyggl",
"axyggl",
"bxyggl",
"cxyggl",
"zwggl",
"axwggl",
"bxwggl",
"cxwggl",
"zglys",
"axglys",
"bxglys",
"cxglys",
"zxygzdl",
"zxygzdl1",
"zxygzdl2",
"zxygzdl3",
"zxygzdl4",
"axwd",
"bxwd",
"cxwd",
"lxwd",
"hjwd",
"zxljl",
"fxljl",
"zhljl",
"ssll",
"bat",
"sn",
"press",
"zxygzdl5",
"meter_no",
"project_id",
"gateway_no",
"meter_type"
],
"connection": [{
"table": [
"sb_fhsj"
],
"jdbcUrl": "jdbc:TAOS-RS://172.16.203.176:6041/db"
}],
"batchSize": 100,
"ignoreTagsUnmatched": true
}
}
}],
"setting": {
"speed": {
"channel": 1
}
}

}
}

迁移到300w的时候,报错java.sql.SQLException: ERROR (2318): Read timed out
过了5分钟后执行又可以了,然后再执行下一个102400的数据,又报错java.sql.SQLException: ERROR (2318): Read timed out

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with bin/datax.py and script/tdengine2.xto3.x.json, then reproduce the migration query using the supplied JDBC URLs and batch settings. Check when the 3-million-row and 102400-row reads time out and compare the successful retry after five minutes. Done requires identifying a reproducible cause and a documented or tested fix for the timeout.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, python
Domain
data-engineering, databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.