向db2数据库入库慢
- Dominant language
- Java
- Stars
- 17.4k
- Forks
- 5.7k
- PR merge metrics
- No merged PRs in 30d
Description
向db2(10.5 fp8)插入数据,无论数据源是csv还是另一个db2库,rdbmswriter速度都很慢,单个channel约1000rec/s. 数据源为db2时, reader端可以跑满千兆带宽,BatchSize=4096,JVM虚机4G, job配置如下:
```
"writer": {
"name": "rdbmswriter",
"parameter": {
"connection": [
{
"jdbcUrl": "jdbc:db2://********",
"table": ["********"]
}
],
"username": "********",
"password": "********",
"table": "********",
"column": [
"STATIS_MONTH",
"SUBS_ID",
"GRP_SUBS_ID",
"PHONE_NO",
"ISPRIMA",
"START_TIME"
],
"#preSql": [
"delete from ********;"
]
}
}
```
多channel下,基本上等于1000*n rec/s,开8个channel时,偶发ERROR=-4229,然后批量插入失败, 变为逐条插入. 单个channel时日志中未发现此情况.
插入速度实在是太慢了,求教解决方法.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start at the rdbmswriter implementation and its batch-insert error handling, using the supplied DB2 JDBC configuration, BatchSize=4096, and single-channel versus eight-channel results as the reproduction. Check DB2 ERROR=-4229 and the fallback to row-by-row inserts; done means explaining the throughput limit and correctly handling the reported batch failure, with a regression check if the project provides one.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100