Error occurred in Compaction Merge thread
- Dominant language
- Java
- Stars
- 6.4k
- Forks
- 1.2k
- Avg merge
- 1d 23h
- Merged PRs (30d)
- 115
Description
**Describe the bug**
iotdbserver报错
**To Reproduce**
Steps to reproduce the behavior:
删除某个时间序列,再新增同名但是类型不同的时间序列
**Expected behavior**
怎么样处理,修复让他不报错。这些报错信息会不会影响正常使用
**Screenshots**
2022-06-06 14:16:18,943 [pool-6-IoTDB-Compaction-10] INFO o.a.i.d.e.c.l.LevelCompactionTsFileManagement:625 - root.ln_1 [Compaction] merge level-0's 6 TsFiles to next level
2022-06-06 14:16:18,943 [pool-6-IoTDB-Compaction-10] INFO o.a.i.d.e.c.l.LevelCompactionTsFileManagement:631 - root.ln_1 [Compaction] start to merge TsFile /opt/soft/apache-iotdb-0.12.4-server-bin/sbin/./../data/data/sequence/root.ln_1/0/0/1652124977275-1041-0-0.tsfile
2022-06-06 14:16:18,944 [pool-6-IoTDB-Compaction-10] INFO o.a.i.d.e.c.l.LevelCompactionTsFileManagement:631 - root.ln_1 [Compaction] start to merge TsFile /opt/soft/apache-iotdb-0.12.4-server-bin/sbin/./../data/data/sequence/root.ln_1/0/0/1652286375972-1048-0-0.tsfile
2022-06-06 14:16:18,944 [pool-6-IoTDB-Compaction-10] INFO o.a.i.d.e.c.l.LevelCompactionTsFileManagement:631 - root.ln_1 [Compaction] start to merge TsFile /opt/soft/apache-iotdb-0.12.4-server-bin/sbin/./../data/data/sequence/root.ln_1/0/0/1652453386626-1053-0-0.tsfile
2022-06-06 14:16:18,944 [pool-6-IoTDB-Compaction-10] INFO o.a.i.d.e.c.l.LevelCompactionTsFileManagement:631 - root.ln_1 [Compaction] start to merge TsFile /opt/soft/apache-iotdb-0.12.4-server-bin/sbin/./../data/data/sequence/root.ln_1/0/0/1652618490908-1058-0-0.tsfile
2022-06-06 14:16:18,944 [pool-6-IoTDB-Compaction-10] INFO o.a.i.d.e.c.l.LevelCompactionTsFileManagement:631 - root.ln_1 [Compaction] start to merge TsFile /opt/soft/apache-iotdb-0.12.4-server-bin/sbin/./../data/data/sequence/root.ln_1/0/0/1652782960971-1065-0-0.tsfile
2022-06-06 14:16:18,944 [pool-6-IoTDB-Compaction-10] INFO o.a.i.d.e.c.l.LevelCompactionTsFileManagement:631 - root.ln_1 [Compaction] start to merge TsFile /opt/soft/apache-iotdb-0.12.4-server-bin/sbin/./../data/data/sequence/root.ln_1/0/0/1652946896255-1074-0-0.tsfile
2022-06-06 14:16:41,441 [pool-6-IoTDB-Compaction-10] WARN o.a.i.t.f.m.s.Statistics:185 - Statistics classes mismatched,no merge: class org.apache.iotdb.tsfile.file.metadata.statistics.BinaryStatistics v.s. class org.apache.iotdb.tsfile.file.metadata.statistics.BooleanStatistics
2022-06-06 14:16:41,442 [pool-6-IoTDB-Compaction-10] INFO o.a.i.d.e.c.l.LevelCompactionTsFileManagement:788 - root.ln_1-0 [Compaction][Restore] Start to restore compaction
2022-06-06 14:16:42,240 [pool-6-IoTDB-Compaction-10] INFO o.a.i.t.r.TsFileSequenceReader:1156 - TsFile /opt/soft/apache-iotdb-0.12.4-server-bin/sbin/./../data/data/sequence/root.ln_1/0/0/1652124977275-1041-1-0.tsfile self-check cannot proceed at position 84028992 recovered, because : reach the end of the file.
2022-06-06 14:16:42,905 [pool-6-IoTDB-Compaction-10] INFO o.a.i.d.e.c.l.LevelCompactionTsFileManagement:805 - root.ln_1-0 [Compaction][Restore] target file /opt/soft/apache-iotdb-0.12.4-server-bin/sbin/./../data/data/sequence/root.ln_1/0/0/1652124977275-1041-1-0.tsfile is incomplete, delete it
2022-06-06 14:16:42,911 [pool-6-IoTDB-Compaction-10] ERROR o.a.i.d.e.c.l.LevelCompactionTsFileManagement:696 - Error occurred in Compaction Merge thread
org.apache.iotdb.tsfile.exception.filter.StatisticsClassException: Statistics classes mismatched: class org.apache.iotdb.tsfile.file.metadata.statistics.BinaryStatistics vs. class org.apache.iotdb.tsfile.file.metadata.statistics.BooleanStatistics
at org.apache.iotdb.tsfile.file.metadata.statistics.Statistics.mergeStatistics(Statistics.java:187)
at org.apache.iotdb.tsfile.file.metadata.ChunkMetadata.mergeChunkMetadata(ChunkMetadata.java:269)
at org.apache.iotdb.db.engine.compaction.utils.CompactionUtils.readByAppendPageMerge(CompactionUtils.java:90)
at org.apache.iotdb.db.engine.compaction.utils.CompactionUtils.writeByAppendPageMerge(CompactionUtils.java:159)
at org.apache.iotdb.db.engine.compaction.utils.CompactionUtils.merge(CompactionUtils.java:400)
at org.apache.iotdb.db.engine.compaction.level.LevelCompactionTsFileManagement.merge(LevelCompactionTsFileManagement.java:638)
at org.apache.iotdb.db.engine.compaction.level.LevelCompactionTsFileManagement.merge(LevelCompactionTsFileManagement.java:526)
at org.apache.iotdb.db.engine.compaction.TsFileManagement$CompactionMergeTask.call(TsFileManagement.java:181)
at org.apache.iotdb.db.engine.storagegroup.StorageGroupProcessor.syncCompactOnePartition(StorageGroupProcessor.java:2002)
at org.apache.iotdb.db.engine.storagegroup.StorageGroupProcessor.access$100(StorageGroupProcessor.java:135)
at org.apache.iotdb.db.engine.storagegroup.StorageGroupProcessor$CompactionAllPartitionTask.call(StorageGroupProcessor.java:524)
at org.apache.iotdb.db.engine.storagegroup.StorageGroupProcessor.merge(StorageGroupProcessor.java:2138)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305)
at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
2022-06-06 14:17:12,912 [pool-6-IoTDB-Compaction-8] INFO o.a.i.d.e.c.l.LevelCompactionTsFileManagement:625 - root.ln_1 [Compaction] merge level-0's 6 TsFiles to next level
2022-06-06 14:17:12,912 [pool-6-IoTDB-Compaction-8] INFO o.a.i.d.e.c.l.LevelCompactionTsFileManagement:631 - root.ln_1 [Compaction] start to merge TsFile /opt/soft/apache-iotdb-0.12.4-server-bin/sbin/./../data/data/sequence/root.ln_1/0/0/1652124977275-1041-0-0.tsfile
2022-06-06 14:17:12,912 [pool-6-IoTDB-Compaction-8] INFO o.a.i.d.e.c.l.LevelCompactionTsFileManagement:631 - root.ln_1 [Compaction] start to merge TsFile /opt/soft/apache-iotdb-0.12.4-server-bin/sbin/./../data/data/sequence/root.ln_1/0/0/1652286375972-1048-0-0.tsfile
2022-06-06 14:17:12,912 [pool-6-IoTDB-Compaction-8] INFO o.a.i.d.e.c.l.LevelCompactionTsFileManagement:631 - root.ln_1 [Compaction] start to merge TsFile /opt/soft/apache-iotdb-0.12.4-server-bin/sbin/./../data/data/sequence/root.ln_1/0/0/1652453386626-1053-0-0.tsfile
2022-06-06 14:17:12,912 [pool-6-IoTDB-Compaction-8] INFO o.a.i.d.e.c.l.LevelCompactionTsFileManagement:631 - root.ln_1 [Compaction] start to merge TsFile /opt/soft/apache-iotdb-0.12.4-server-bin/sbin/./../data/data/sequence/root.ln_1/0/0/1652618490908-1058-0-0.tsfile
2022-06-06 14:17:12,912 [pool-6-IoTDB-Compaction-8] INFO o.a.i.d.e.c.l.LevelCompactionTsFileManagement:631 - root.ln_1 [Compaction] start to merge TsFile /opt/soft/apache-iotdb-0.12.4-server-bin/sbin/./../data/data/sequence/root.ln_1/0/0/1652782960971-1065-0-0.tsfile
2022-06-06 14:17:12,912 [pool-6-IoTDB-Compaction-8] INFO o.a.i.d.e.c.l.LevelCompactionTsFileManagement:631 - root.ln_1 [Compaction] start to merge TsFile /opt/soft/apache-iotdb-0.12.4-server-bin/sbin/./../data/data/sequence/root.ln_1/0/0/1652946896255-1074-0-0.tsfile
2022-06-06 14:17:35,421 [pool-6-IoTDB-Compaction-8] WARN o.a.i.t.f.m.s.Statistics:185 - Statistics classes mismatched,no merge: class org.apache.iotdb.tsfile.file.metadata.statistics.BinaryStatistics v.s. class org.apache.iotdb.tsfile.file.metadata.statistics.BooleanStatistics
2022-06-06 14:17:35,421 [pool-6-IoTDB-Compaction-8] INFO o.a.i.d.e.c.l.LevelCompactionTsFileManagement:788 - root.ln_1-0 [Compaction][Restore] Start to restore compaction
2022-06-06 14:17:36,210 [pool-6-IoTDB-Compaction-8] INFO o.a.i.t.r.TsFileSequenceReader:1156 - TsFile /opt/soft/apache-iotdb-0.12.4-server-bin/sbin/./../data/data/sequence/root.ln_1/0/0/1652124977275-1041-1-0.tsfile self-check cannot proceed at position 84028992 recovered, because : reach the end of the file.
2022-06-06 14:17:36,877 [pool-6-IoTDB-Compaction-8] INFO o.a.i.d.e.c.l.LevelCompactionTsFileManagement:805 - root.ln_1-0 [Compaction][Restore] target file /opt/soft/apache-iotdb-0.12.4-server-bin/sbin/./../data/data/sequence/root.ln_1/0/0/1652124977275-1041-1-0.tsfile is incomplete, delete it
2022-06-06 14:17:36,884 [pool-6-IoTDB-Compaction-8] ERROR o.a.i.d.e.c.l.LevelCompactionTsFileManagement:696 - Error occurred in Compaction Merge thread
org.apache.iotdb.tsfile.exception.filter.StatisticsClassException: Statistics classes mismatched: class org.apache.iotdb.tsfile.file.metadata.statistics.BinaryStatistics vs. class org.apache.iotdb.tsfile.file.metadata.statistics.BooleanStatistics
at org.apache.iotdb.tsfile.file.metadata.statistics.Statistics.mergeStatistics(Statistics.java:187)
at org.apache.iotdb.tsfile.file.metadata.ChunkMetadata.mergeChunkMetadata(ChunkMetadata.java:269)
at org.apache.iotdb.db.engine.compaction.utils.CompactionUtils.readByAppendPageMerge(CompactionUtils.java:90)
at org.apache.iotdb.db.engine.compaction.utils.CompactionUtils.writeByAppendPageMerge(CompactionUtils.java:159)
at org.apache.iotdb.db.engine.compaction.utils.CompactionUtils.merge(CompactionUtils.java:400)
at org.apache.iotdb.db.engine.compaction.level.LevelCompactionTsFileManagement.merge(LevelCompactionTsFileManagement.java:638)
at org.apache.iotdb.db.engine.compaction.level.LevelCompactionTsFileManagement.merge(LevelCompactionTsFileManagement.java:526)
at org.apache.iotdb.db.engine.compaction.TsFileManagement$CompactionMergeTask.call(TsFileManagement.java:181)
at org.apache.iotdb.db.engine.storagegroup.StorageGroupProcessor.syncCompactOnePartition(StorageGroupProcessor.java:2002)
at org.apache.iotdb.db.engine.storagegroup.StorageGroupProcessor.access$100(StorageGroupProcessor.java:135)
at org.apache.iotdb.db.engine.storagegroup.StorageGroupProcessor$CompactionAllPartitionTask.call(StorageGroupProcessor.java:524)
at org.apache.iotdb.db.engine.storagegroup.StorageGroupProcessor.merge(StorageGroupProcessor.java:2138)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305)
at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
2022-06-06 14:17:48,465 [pool-18-IoTDB-RPC-Client-20] INFO o.a.i.d.s.TSServiceImpl:276 - Session-20 is closing
2022-06-06 14:18:06,884 [pool-6-IoTDB-Compaction-4] INFO o.a.i.d.e.c.l.LevelCompactionTsFileManagement:625 - root.ln_1 [Compaction] merge level-0's 6 TsFiles to next level
2022-06-06 14:18:06,885 [pool-6-IoTDB-Compaction-4] INFO o.a.i.d.e.c.l.LevelCompactionTsFileManagement:631 - root.ln_1 [Compaction] start to merge TsFile /opt/soft/apache-iotdb-0.12.4-server-bin/sbin/./../data/data/sequence/root.ln_1/0/0/1652124977275-1041-0-0.tsfile
2022-06-06 14:18:06,885 [pool-6-IoTDB-Compaction-4] INFO o.a.i.d.e.c.l.LevelCompactionTsFileManagement:631 - root.ln_1 [Compaction] start to merge TsFile /opt/soft/apache-iotdb-0.12.4-server-bin/sbin/./../data/data/sequence/root.ln_1/0/0/1652286375972-1048-0-0.tsfile
2022-06-06 14:18:06,885 [pool-6-IoTDB-Compaction-4] INFO o.a.i.d.e.c.l.LevelCompactionTsFileManagement:631 - root.ln_1 [Compaction] start to merge TsFile /opt/soft/apache-iotdb-0.12.4-server-bin/sbin/./../data/data/sequence/root.ln_1/0/0/1652453386626-1053-0-0.tsfile
2022-06-06 14:18:06,885 [pool-6-IoTDB-Compaction-4] INFO o.a.i.d.e.c.l.LevelCompactionTsFileManagement:631 - root.ln_1 [Compaction] start to merge TsFile /opt/soft/apache-iotdb-0.12.4-server-bin/sbin/./../data/data/sequence/root.ln_1/0/0/1652618490908-1058-0-0.tsfile
2022-06-06 14:18:06,885 [pool-6-IoTDB-Compaction-4] INFO o.a.i.d.e.c.l.LevelCompactionTsFileManagement:631 - root.ln_1 [Compaction] start to merge TsFile /opt/soft/apache-iotdb-0.12.4-server-bin/sbin/./../data/data/sequence/root.ln_1/0/0/1652782960971-1065-0-0.tsfile
2022-06-06 14:18:06,885 [pool-6-IoTDB-Compaction-4] INFO o.a.i.d.e.c.l.LevelCompactionTsFileManagement:631 - root.ln_1 [Compaction] start to merge TsFile /opt/soft/apache-iotdb-0.12.4-server-bin/sbin/./../data/data/sequence/root.ln_1/0/0/1652946896255-1074-0-0.tsfile
**Desktop (please complete the following information):**
- OS: linux
- Browser 无
- Version 0.12.4
**Additional context**
Add any other context about the problem here.
Contributor guide
Research direction
Start with Statistics.mergeStatistics and ChunkMetadata.mergeChunkMetadata, then trace the call through CompactionUtils.readByAppendPageMerge and LevelCompactionTsFileManagement.merge. Reproduce the sequence of deleting a time series and recreating it with a different type in IoTDB 0.12.4. Done means compaction no longer raises StatisticsClassException and the affected data remains usable.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100