apache / apache/iotdb

ioTDB运行一段时间报这个错误是什么原因 org.apache.iotdb.db.exception.BatchProcessException: Batch process failed:[TSStatus(code:411, message:Batch process failed:[TSStatus(code:413)])

オープン
#8,371 コメント 4 件 リアクション 0 件 担当者 0 名 GitHub で見る
主要言語
Java
スター
6.4k
フォーク
1.2k
平均マージ
1日 23時間
マージ済み PR(30日)
115

説明

### Search before asking

- [X] I searched in the [issues](https://github.com/apache/iotdb/issues) and found nothing similar.

### Version

0.13.3

### Describe the bug and provide the minimal reproduce step

---

### What did you expect to see?

运行一段时间后出现以下异常:
2022-12-07 05:14:00,167 [pool-6-IoTDB-Flush-6] INFO o.a.i.d.e.f.MemTableFlushTask:187 - Storage group root.tb/0 memtable PrimitiveMemTable{planIndex=[0,0]} flushing a memtable has finished! Time consumption: 5ms
2022-12-07 05:14:00,170 [pool-6-IoTDB-Flush-6] INFO o.a.i.d.w.n.ExclusiveWriteLogNode:273 - Log node root.tb/0-1670357048721-73-0-0.tsfile cleaned old file
2022-12-07 05:14:00,170 [pool-6-IoTDB-Flush-6] WARN o.a.i.d.rescon.SystemInfo:152 - SG (root.tb) released memory (delta: -13237666), but system is still in reject status (totalSgMemCost: 832284344).
2022-12-07 05:14:00,172 [pool-6-IoTDB-Flush-6] INFO o.a.i.d.e.s.TsFileProcessor:1206 - The compression ratio of tsfile /iotdb/sbin/../data/data/unsequence/root.tb/0/0/1670357048721-73-0-0.tsfile is 85.33333333333333, totalMemTableSize: 53760, the file size: 630
2022-12-07 05:14:00,173 [pool-6-IoTDB-Flush-6] INFO o.a.i.d.c.a.CompressionRatio:98 - Compression ratio is 78.80125284761064
2022-12-07 05:14:00,174 [pool-6-IoTDB-Flush-6] INFO o.a.i.d.e.s.TsFileProcessor:1224 - Start to end file file is /iotdb/sbin/../data/data/unsequence/root.tb/0/0/1670357048721-73-0-0.tsfile, status: UNCLOSED
2022-12-07 05:14:00,175 [pool-6-IoTDB-Flush-6] INFO o.a.i.d.e.s.TsFileProcessor:1228 - Ended file file is /iotdb/sbin/../data/data/unsequence/root.tb/0/0/1670357048721-73-0-0.tsfile, status: UNCLOSED
2022-12-07 05:14:00,182 [pool-6-IoTDB-Flush-6] INFO o.a.i.d.e.s.VirtualStorageGroupProcessor:2214 - signal closing storage group condition in root.tb-0
2022-12-07 05:14:00,183 [pool-6-IoTDB-Flush-6] WARN o.a.i.d.rescon.SystemInfo:152 - SG (root.tb) released memory (delta: 1896), but system is still in reject status (totalSgMemCost: 832282448).
2022-12-07 05:14:00,183 [pool-6-IoTDB-Flush-6] INFO o.a.i.d.e.s.TsFileProcessor:1242 - Storage group root.tb/0 close the file /iotdb/sbin/../data/data/unsequence/root.tb/0/0/1670357048721-73-0-0.tsfile, TsFile size is 1605, time consumption of flushing metadata is 9ms
2022-12-07 05:14:05,816 [pool-20-IoTDB-Compaction_Schedule-root.tb-0-1] WARN o.a.i.d.e.c.c.r.s.RewriteCompactionFileSelector:322 - /iotdb/sbin/../data/data/sequence/root.tb/0/0/1669887999251-7-0-61.tsfile is selected for cross space compaction, it is overlapped with /iotdb/sbin/../data/data/unsequence/root.tb/0/0/1670357048721-73-0-0.tsfile. It's selected because its end time 1670097600000 is greater than or equals to unseq file's endTime 1669852800000 in device root.tb.3ab8290000bd11ed9409d76c0e04bbb5
2022-12-07 05:14:05,816 [pool-20-IoTDB-Compaction_Schedule-root.tb-0-1] WARN o.a.i.d.e.c.c.r.s.RewriteCompactionFileSelector:322 - /iotdb/sbin/../data/data/sequence/root.tb/0/0/1669887999251-7-0-61.tsfile is selected for cross space compaction, it is overlapped with /iotdb/sbin/../data/data/unsequence/root.tb/0/0/1670357048721-73-0-0.tsfile. It's selected because its end time 1670097600000 is greater than or equals to unseq file's endTime 1669852800000 in device root.tb.4034092000eb11edbe82a163009b23f6
2022-12-07 05:14:05,825 [pool-20-IoTDB-Compaction_Schedule-root.tb-0-1] INFO o.a.i.d.e.c.c.r.s.RewriteCompactionFileSelector:136 - Selected merge candidates, 1 seqFiles, 1 unseqFiles, total memory cost 1473960, time consumption 10ms
2022-12-07 05:14:05,825 [pool-20-IoTDB-Compaction_Schedule-root.tb-0-1] INFO o.a.i.d.e.c.c.r.RewriteCrossSpaceCompactionSelector:116 - select files for cross compaction, sequence files: [file is /iotdb/sbin/../data/data/sequence/root.tb/0/0/1669887999251-7-0-61.tsfile, status: CLOSED], unsequence files [file is /iotdb/sbin/../data/data/unsequence/root.tb/0/0/1670357048721-73-0-0.tsfile, status: CLOSED], memory cost is 1473960
2022-12-07 05:14:05,826 [pool-20-IoTDB-Compaction_Schedule-root.tb-0-1] INFO o.a.i.d.e.c.c.r.RewriteCrossSpaceCompactionSelector:133 - root.tb-0 [Compaction] submit a task with 1 sequence file and 1 unseq files
2022-12-07 05:14:05,827 [pool-9-IoTDB-Compaction-8] INFO o.a.i.d.e.c.c.r.t.RewriteCrossSpaceCompactionTask:149 - root.tb-0 [Compaction] CrossSpaceCompactionTask start. Sequence files : [file is /iotdb/sbin/../data/data/sequence/root.tb/0/0/1669887999251-7-0-61.tsfile, status: COMPACTING], unsequence files : [file is /iotdb/sbin/../data/data/unsequence/root.tb/0/0/1670357048721-73-0-0.tsfile, status: COMPACTING], seq files size is 64.20209980010986 MB, unseq file size is 0.0015306472778320312 MB, total size is 64.2036304473877 MB
2022-12-07 05:14:10,257 [pool-17-IoTDB-RPC-Client-11] WARN o.a.i.d.u.ErrorHandlingUtils:146 - Exception occurred while processing non-query.
org.apache.iotdb.db.exception.BatchProcessException: Batch process failed:[TSStatus(code:411, message:Batch process failed:[TSStatus(code:413)])]
at org.apache.iotdb.db.qp.executor.PlanExecutor.insertTabletSerial(PlanExecutor.java:1936)
at org.apache.iotdb.db.qp.executor.PlanExecutor.insertTablet(PlanExecutor.java:1916)
at org.apache.iotdb.db.qp.executor.PlanExecutor.processNonQuery(PlanExecutor.java:306)
at org.apache.iotdb.db.service.basic.StandaloneServiceProvider.executeNonQuery(StandaloneServiceProvider.java:54)
at org.apache.iotdb.db.service.thrift.impl.TSServiceImpl.executeNonQueryPlan(TSServiceImpl.java:2202)
at org.apache.iotdb.db.service.thrift.impl.TSServiceImpl.insertTabletsInternally(TSServiceImpl.java:1699)
at org.apache.iotdb.db.service.thrift.impl.TSServiceImpl.insertTablets(TSServiceImpl.java:1649)
at org.apache.iotdb.service.rpc.thrift.TSIService$Processor$insertTablets.getResult(TSIService.java:3626)
at org.apache.iotdb.service.rpc.thrift.TSIService$Processor$insertTablets.getResult(TSIService.java:3606)
at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:38)
at org.apache.iotdb.db.service.thrift.ProcessorWithMetrics.process(ProcessorWithMetrics.java:64)
at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:248)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.base/java.lang.Thread.run(Unknown Source)

### What did you see instead?

---

### Anything else?

_No response_

### Are you willing to submit a PR?

- [X] I'm willing to submit a PR!

コントリビューションガイド

コントリビューションガイドを開く

調査の方向性

PlanExecutor.java の insertTabletSerial(1936 行目付近)から開始し、ステータスコード 411 と 413 を挿入パスに沿って追跡します。関連する SystemInfo のメモリ警告と compaction のログエントリを確認し、IoTDB 0.13.3 で長時間実行される挿入シナリオを再現します。BatchProcessException の原因と適用可能な解決策が確立されれば完了です。

索引モデルが issue の本文から書いたものです。

評価

技術スタック
java
領域
databases
issue の種類
バグ
難易度
4/5
見積もり時間
3〜5日
活発さ
停滞
明瞭さ
説明が足りない
初心者へのやさしさ
20/100

新しい issue をメールで受け取る

初心者向けの GitHub issue を短くまとめたダイジェスト。