apache / apache/iotdb

inserting data slow when insert data in bulk

Aperta
#8,871 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub
Lingua principale
Java
Stelle
6.4k
Fork
1.2k
Merge medio
1g 23h
PR unite (30g)
115

Descrizione

**environment:**
version: 1.0.0
deployment: all in one, one confignode, one datanode
host: cpu: 16core, mem:16G, disk: ssd, os: ubuntu 22.04
client sdk: python, use normal tablet

**background:**
the measurements number of a batch of data is huge, in my usage scenario, sometimes it is 200,000.

**experiment:**
I tested the insertion performance.
Insertion takes 2.5s-4s when all measurement points are inserted in one database and the measurements number of a batch is 200,000.
Insertion takes 0.3s when all measurement points are inserted in 10 databases of one instance and 10 threads, one thread insert 20,000 measurements once.

I looked at the usage performance of host, cpu only used under 200%, memory is still remain, disk io is 8MB/s.
None of this is a bottleneck. so i strace the process:

```
% time seconds usecs/call calls errors syscall
------ ----------- ----------- --------- --------- ------------------
79.04 52.355396 1545 33867 4478 futex
8.88 5.880482 2816 2088 epoll_pwait2
5.25 3.480728 102374 34 15 restart_syscall
4.60 3.049973 7682 397 epoll_wait
1.84 1.217815 1363 893 recvfrom
0.19 0.124981 17 7105 writev
0.10 0.065069 10 6045 read
0.03 0.023011 442 52 pwrite64
0.02 0.013909 5 2750 write
0.01 0.005350 137 39 fdatasync
```

in strace log, many lock timeout:
```
335339 16:00:27.993273 futex(0x7f1b240f6028, FUTEX_WAKE_PRIVATE, 1) = 0 <0.000011>
335339 16:00:27.993317 futex(0x7f1b240f6078, FUTEX_WAIT_BITSET_PRIVATE, 0, {tv_sec=359621, tv_nsec=748782361}, FUTEX_BITSET_MATCH_ANY
335381 16:00:27.995534 <... futex resumed>) = -1 ETIMEDOUT (Connection timed out) <0.200098>
335381 16:00:27.995648 futex(0x7f19e0025718, FUTEX_WAKE_PRIVATE, 1) = 0 <0.000100>
335381 16:00:27.995913 futex(0x7f19e0025768, FUTEX_WAIT_BITSET_PRIVATE, 0, {tv_sec=359621, tv_nsec=901319853}, FUTEX_BITSET_MATCH_ANY
335454 16:00:27.996630 <... futex resumed>) = -1 ETIMEDOUT (Connection timed out) <0.100112>
335454 16:00:27.996656 futex(0x7f19e00b70d8, FUTEX_WAKE_PRIVATE, 1) = 0 <0.000009>
335454 16:00:27.996762 futex(0x7f19e00b7128, FUTEX_WAIT_BITSET_PRIVATE, 0, {tv_sec=359621, tv_nsec=802178374}, FUTEX_BITSET_MATCH_ANY
```

so i traceing java process use jstack, but I can't read the contents, but there is code related to locks.

**question:**

1. In cases where there are so many measurement points, How do I need to optimize it? I look at the documentation and there are too many configuration parameters. https://iotdb.apache.org/UserGuide/Master/Reference/Common-Config-Manual.html
2. Can it improve performance, if i use cluster mode.

Guida per i contributori

Apri la guida per i contributori

Direzione di ricerca

Il report non indica file o test del repository; inizia riproducendo l’inserimento bulk con il deployment IoTDB indicato, il client Python, l’output di strace e il capture di jstack. Leggi il Common Config Manual collegato e confronta le esecuzioni con un singolo database e in modalità cluster. Il lavoro è completo quando vengono identificati il lock o la configurazione limitante e viene documentata un’ottimizzazione verificata o un risultato sulle prestazioni del cluster.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Valutazione

Stack tecnologico
java, python
Ambito
databases, performance
Tipo di issue
Bug
Difficoltà
4/5
Tempo stimato
3-5 giorni
Stato di attività
Ferma
Chiarezza
Da chiarire
Idoneità per principianti
25/100

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.