迁移tdengine2.1.3.0版本到tgengine3.2.3.0报错java.sql.SQLException: ERROR (2318): Read timed out
- Dominant language
- Java
- Stars
- 17.4k
- Forks
- 5.7k
- PR merge metrics
- No merged PRs in 30d
Description
迁移tdengine2.1.3.0版本-->3.2.3.0
使用datax工具,python bin/datax.py --jvm="-Xms4G -Xmx4G" script/tdengine2.xto3.x.json
script/tdengine2.xto3.x.json
{
"job": {
"content": [{
"reader": {
"name": "tdengine20reader",
"parameter": {
"username": "root",
"password": "taosdata",
"connection": [{
"querySql": [
"select tbname,* from db.sb_fhsj where sjsj >= '2024-01-01 00:00:00' and sjsj < '2024-02-01 00:00:00' limit 3788801,3891200"
],
"jdbcUrl": [
"jdbc:TAOS-RS://172.16.203.172:6041/db?timestampFormat=TIMESTAMP"
]
}]
}
},
"writer": {
"name": "tdengine30writer",
"parameter": {
"username": "root",
"password": "*",
"column": [
"tbname",
"sjsj",
"jssj",
"yxzt",
"sjzt",
"jdqzt",
"axdy",
"bxdy",
"cxdy",
"axdl",
"bxdl",
"cxdl",
"sydl",
"zyggl",
"axyggl",
"bxyggl",
"cxyggl",
"zwggl",
"axwggl",
"bxwggl",
"cxwggl",
"zglys",
"axglys",
"bxglys",
"cxglys",
"zxygzdl",
"zxygzdl1",
"zxygzdl2",
"zxygzdl3",
"zxygzdl4",
"axwd",
"bxwd",
"cxwd",
"lxwd",
"hjwd",
"zxljl",
"fxljl",
"zhljl",
"ssll",
"bat",
"sn",
"press",
"zxygzdl5",
"meter_no",
"project_id",
"gateway_no",
"meter_type"
],
"connection": [{
"table": [
"sb_fhsj"
],
"jdbcUrl": "jdbc:TAOS-RS://172.16.203.176:6041/db"
}],
"batchSize": 100,
"ignoreTagsUnmatched": true
}
}
}],
"setting": {
"speed": {
"channel": 1
}
}
}
}
迁移到300w的时候,报错java.sql.SQLException: ERROR (2318): Read timed out
过了5分钟后执行又可以了,然后再执行下一个102400的数据,又报错java.sql.SQLException: ERROR (2318): Read timed out
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with bin/datax.py and script/tdengine2.xto3.x.json, then reproduce the migration query using the supplied JDBC URLs and batch settings. Check when the 3-million-row and 102400-row reads time out and compare the successful retry after five minutes. Done requires identifying a reproducible cause and a documented or tested fix for the timeout.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java, python
- Domain
- data-engineering, databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100