乱序数据文件导致iotdb server 内存暴涨,无法查询数据
- Langage dominant
- Java
- Étoiles
- 6.4k
- Forks
- 1.2k
- Merge moyen
- 1 j 23 h
- PR mergées (30 j)
- 115
Description
iotdb 版本:0.13.0
设备不写入数据的情况下,我启动了iotdb server,如下现象:
1,查数据,出不来,iotdb server 报了null 错误;
2、电脑中的java 进程一直涨,我的电脑很快就占用到4G了。。
然后我就是怀疑,iotdb在处理之前的数据,就去看data目录,发现乱序数据目录下有文件,我的场景下不应该出现乱序数据。。所以就删了,就恢复正常了。

日志如下:
2022-11-12 15:04:17,679 [pool-7-IoTDB-Compaction-1] INFO o.a.i.d.e.c.c.CrossSpaceCompactionExceptionHandler:58 - [Compaction][ExceptionHandler] Cross space compaction start handling exception, source seqFiles is [file is D:\apache-iotdb-0.13.0-all-bin\data\data\sequence\root.rczz\0\0\1668163631223-2-0-0.tsfile, compactionCandidate: true, compacting: true], source unseqFiles is [file is D:\apache-iotdb-0.13.0-all-bin\data\data\unsequence\root.rczz\0\0\1668216201890-5-0-0.tsfile, compactionCandidate: true, compacting: true]
2022-11-12 15:04:17,685 [pool-7-IoTDB-Compaction-1] ERROR o.a.i.d.e.c.t.AbstractCompactionTask:70 - null
java.lang.NullPointerException: null
at org.apache.iotdb.db.engine.compaction.inner.utils.MultiTsFileDeviceIterator.hasNextDevice(MultiTsFileDeviceIterator.java:97)
at org.apache.iotdb.db.engine.compaction.CompactionUtils.compact(CompactionUtils.java:92)
at org.apache.iotdb.db.engine.compaction.cross.rewrite.task.RewriteCrossSpaceCompactionTask.executeCompaction(RewriteCrossSpaceCompactionTask.java:145)
at org.apache.iotdb.db.engine.compaction.cross.rewrite.task.RewriteCrossSpaceCompactionTask.doCompaction(RewriteCrossSpaceCompactionTask.java:87)
at org.apache.iotdb.db.engine.compaction.task.AbstractCompactionTask.call(AbstractCompactionTask.java:68)
at org.apache.iotdb.db.engine.compaction.task.AbstractCompactionTask.call(AbstractCompactionTask.java:45)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
2022-11-12 15:04:52,630 [pool-17-IoTDB-RPC-Client-1] INFO o.a.i.d.s.b.ServiceProvider:188 - IoTDB: Login status: Login successfully. User : root, opens Session-1
2022-11-12 15:04:52,821 [pool-17-IoTDB-RPC-Client-1] INFO o.a.i.d.c.IoTDBThreadPoolFactory:68 - new fixed thread pool: Query, thread number: 12
2022-11-12 15:04:56,346 [pool-17-IoTDB-RPC-Client-2] INFO o.a.i.d.s.b.ServiceProvider:188 - IoTDB: Login status: Login successfully. User : root, opens Session-2
2022-11-12 15:04:57,495 [pool-3-IoTDB-timedQuerySqlCount-1] INFO o.a.i.d.s.b.QueryFrequencyRecorder:42 - Query count in current 1 minute 2
2022-11-12 15:05:08,317 [pool-17-IoTDB-RPC-Client-3] INFO o.a.i.d.s.b.ServiceProvider:188 - IoTDB: Login status: Login successfully. User : root, opens Session-3
2022-11-12 15:05:44,731 [pool-17-IoTDB-RPC-Client-4] INFO o.a.i.d.s.b.ServiceProvider:188 - IoTDB: Login status: Login successfully. User : root, opens Session-4
Guide de contribution
Ouvrir le guide de contribution
Piste de recherche
Commencez par la trace de la pile à MultiTsFileDeviceIterator.java:97 et suivez ses appelants dans CompactionUtils.java et RewriteCrossSpaceCompactionTask.java. Reproduisez l’échec avec les fichiers .tsfile de sequence et unsequence signalés, puis vérifiez que les requêtes du serveur n’échouent plus avec une NullPointerException et que la mémoire ne continue pas à augmenter pendant la compaction.
Rédigé par le modèle d'indexation à partir du texte de l'issue.
Évaluation
- Stack technique
- java
- Domaine
- databases
- Type d'issue
- Bug
- Difficulté
- 4/5
- Temps estimé
- 3-5 jours
- Activité
- À l'abandon
- Clarté
- Plutôt claire
- Accessibilité débutants
- 35/100