apache / apache/iotdb

inserting data slow when insert data in bulk

未关闭
#8,871 0 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看
主要语言
Java
星标
6.4k
派生
1.2k
平均合并
1 天 23 小时
30 天内合并 PR
115

描述

**environment:**
version: 1.0.0
deployment: all in one, one confignode, one datanode
host: cpu: 16core, mem:16G, disk: ssd, os: ubuntu 22.04
client sdk: python, use normal tablet

**background:**
the measurements number of a batch of data is huge, in my usage scenario, sometimes it is 200,000.

**experiment:**
I tested the insertion performance.
Insertion takes 2.5s-4s when all measurement points are inserted in one database and the measurements number of a batch is 200,000.
Insertion takes 0.3s when all measurement points are inserted in 10 databases of one instance and 10 threads, one thread insert 20,000 measurements once.

I looked at the usage performance of host, cpu only used under 200%, memory is still remain, disk io is 8MB/s.
None of this is a bottleneck. so i strace the process:

```
% time seconds usecs/call calls errors syscall
------ ----------- ----------- --------- --------- ------------------
79.04 52.355396 1545 33867 4478 futex
8.88 5.880482 2816 2088 epoll_pwait2
5.25 3.480728 102374 34 15 restart_syscall
4.60 3.049973 7682 397 epoll_wait
1.84 1.217815 1363 893 recvfrom
0.19 0.124981 17 7105 writev
0.10 0.065069 10 6045 read
0.03 0.023011 442 52 pwrite64
0.02 0.013909 5 2750 write
0.01 0.005350 137 39 fdatasync
```

in strace log, many lock timeout:
```
335339 16:00:27.993273 futex(0x7f1b240f6028, FUTEX_WAKE_PRIVATE, 1) = 0 <0.000011>
335339 16:00:27.993317 futex(0x7f1b240f6078, FUTEX_WAIT_BITSET_PRIVATE, 0, {tv_sec=359621, tv_nsec=748782361}, FUTEX_BITSET_MATCH_ANY
335381 16:00:27.995534 <... futex resumed>) = -1 ETIMEDOUT (Connection timed out) <0.200098>
335381 16:00:27.995648 futex(0x7f19e0025718, FUTEX_WAKE_PRIVATE, 1) = 0 <0.000100>
335381 16:00:27.995913 futex(0x7f19e0025768, FUTEX_WAIT_BITSET_PRIVATE, 0, {tv_sec=359621, tv_nsec=901319853}, FUTEX_BITSET_MATCH_ANY
335454 16:00:27.996630 <... futex resumed>) = -1 ETIMEDOUT (Connection timed out) <0.100112>
335454 16:00:27.996656 futex(0x7f19e00b70d8, FUTEX_WAKE_PRIVATE, 1) = 0 <0.000009>
335454 16:00:27.996762 futex(0x7f19e00b7128, FUTEX_WAIT_BITSET_PRIVATE, 0, {tv_sec=359621, tv_nsec=802178374}, FUTEX_BITSET_MATCH_ANY
```

so i traceing java process use jstack, but I can't read the contents, but there is code related to locks.

**question:**

1. In cases where there are so many measurement points, How do I need to optimize it? I look at the documentation and there are too many configuration parameters. https://iotdb.apache.org/UserGuide/Master/Reference/Common-Config-Manual.html
2. Can it improve performance, if i use cluster mode.

贡献指南

打开贡献指南

调研方向

报告未提及任何仓库文件或测试;首先使用所述的 IoTDB 部署、Python 客户端、strace 输出和 jstack 捕获来复现批量插入。阅读链接的 Common Config Manual,并比较单数据库运行和集群模式运行。完成的标准是确定限制因素所在的锁或配置,并记录经过验证的优化或集群性能结果。

由索引模型根据 Issue 内容生成。

评估

技术栈
java, python
领域
databases, performance
Issue 类型
缺陷
难度
4/5
预计耗时
3-5 天
活跃度
停滞
描述清晰度
需要澄清
新手友好度
25/100

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。