influxdata / influxdata/influxdb
Every 2-3 days "Aborted compaction"
- Dominant language
- Rust
- Stars
- 31.7k
- Forks
- 3.7k
- Avg merge
- 13h 37m
- Merged PRs (30d)
- 8
Description
__Steps to reproduce:__
I don't have any idea how to reproduce it.
__Actual behavior:__
My InfluxDB is running on Raspberry Pi. 2-10 Raspis are writing data into it. After 2-4 days I have this problem:
Aborted compaction
I switched from TSM to TSI. But the problem still exist. And now I lost new data (whole week).
Why I have so many problems? How can I solve it? I can't run SQL DB on the Raspberry Pi.
__Environment info:__
* System info: Linux pi-server 4.19.75-v7+ #1270 SMP Tue Sep 24 18:45:11 BST 2019 armv7l GNU/Linux
* InfluxDB version: 1.7.9
* Other relevant environment details: SQLlite
__Logs:__
Dec 2 00:21:03 pi-server influxd[1321]: ts=2019-12-01T23:21:03.628706Z lvl=info msg=“TSM compaction (end)” log_id=0JSvpwlG000 engine=tsm1 tsm1_strategy=full tsm1_optimize=false trace_id=0JSx7Pdl000 op_name=tsm1_compact_group op_event=end op_elapsed=1001.551ms
Dec 2 00:21:04 pi-server influxd[1321]: ts=2019-12-01T23:21:04.627120Z lvl=info msg=“TSM compaction (start)” log_id=0JSvpwlG000 engine=tsm1 tsm1_strategy=full tsm1_optimize=false trace_id=0JSx7XSl000 op_name=tsm1_compact_group op_event=start
Dec 2 00:21:04 pi-server influxd[1321]: ts=2019-12-01T23:21:04.627250Z lvl=info msg=“Beginning compaction” log_id=0JSvpwlG000 engine=tsm1 tsm1_strategy=full tsm1_optimize=false trace_id=0JSx7XSl000 op_name=tsm1_compact_group tsm1_files_n=10
Dec 2 00:21:04 pi-server influxd[1321]: ts=2019-12-01T23:21:04.627310Z lvl=info msg=“Compacting file” log_id=0JSvpwlG000 engine=tsm1 tsm1_strategy=full tsm1_optimize=false trace_id=0JSx7XSl000 op_name=tsm1_compact_group tsm1_index=0 tsm1_file=/var/lib/influxdb/data/telegraf/autogen/93/000000032-000000003.tsm
Dec 2 00:21:04 pi-server influxd[1321]: ts=2019-12-01T23:21:04.627374Z lvl=info msg=“Compacting file” log_id=0JSvpwlG000 engine=tsm1 tsm1_strategy=full tsm1_optimize=false trace_id=0JSx7XSl000 op_name=tsm1_compact_group tsm1_index=1 tsm1_file=/var/lib/influxdb/data/telegraf/autogen/93/000000064-000000003.tsm
Dec 2 00:21:04 pi-server influxd[1321]: ts=2019-12-01T23:21:04.627436Z lvl=info msg=“Compacting file” log_id=0JSvpwlG000 engine=tsm1 tsm1_strategy=full tsm1_optimize=false trace_id=0JSx7XSl000 op_name=tsm1_compact_group tsm1_index=2 tsm1_file=/var/lib/influxdb/data/telegraf/autogen/93/000000072-000000002.tsm
Dec 2 00:21:04 pi-server influxd[1321]: ts=2019-12-01T23:21:04.627497Z lvl=info msg=“Compacting file” log_id=0JSvpwlG000 engine=tsm1 tsm1_strategy=full tsm1_optimize=false trace_id=0JSx7XSl000 op_name=tsm1_compact_group tsm1_index=3 tsm1_file=/var/lib/influxdb/data/telegraf/autogen/93/000000081-000000002.tsm
Dec 2 00:21:04 pi-server influxd[1321]: ts=2019-12-01T23:21:04.627559Z lvl=info msg=“Compacting file” log_id=0JSvpwlG000 engine=tsm1 tsm1_strategy=full tsm1_optimize=false trace_id=0JSx7XSl000 op_name=tsm1_compact_group tsm1_index=4 tsm1_file=/var/lib/influxdb/data/telegraf/autogen/93/000000089-000000002.tsm
Dec 2 00:21:04 pi-server influxd[1321]: ts=2019-12-01T23:21:04.627620Z lvl=info msg=“Compacting file” log_id=0JSvpwlG000 engine=tsm1 tsm1_strategy=full tsm1_optimize=false trace_id=0JSx7XSl000 op_name=tsm1_compact_group tsm1_index=5 tsm1_file=/var/lib/influxdb/data/telegraf/autogen/93/000000090-000000001.tsm
Dec 2 00:21:04 pi-server influxd[1321]: ts=2019-12-01T23:21:04.627681Z lvl=info msg=“Compacting file” log_id=0JSvpwlG000 engine=tsm1 tsm1_strategy=full tsm1_optimize=false trace_id=0JSx7XSl000 op_name=tsm1_compact_group tsm1_index=6 tsm1_file=/var/lib/influxdb/data/telegraf/autogen/93/000000091-000000001.tsm
Dec 2 00:21:04 pi-server influxd[1321]: ts=2019-12-01T23:21:04.627742Z lvl=info msg=“Compacting file” log_id=0JSvpwlG000 engine=tsm1 tsm1_strategy=full tsm1_optimize=false trace_id=0JSx7XSl000 op_name=tsm1_compact_group tsm1_index=7 tsm1_file=/var/lib/influxdb/data/telegraf/autogen/93/000000092-000000001.tsm
Dec 2 00:21:04 pi-server influxd[1321]: ts=2019-12-01T23:21:04.627803Z lvl=info msg=“Compacting file” log_id=0JSvpwlG000 engine=tsm1 tsm1_strategy=full tsm1_optimize=false trace_id=0JSx7XSl000 op_name=tsm1_compact_group tsm1_index=8 tsm1_file=/var/lib/influxdb/data/telegraf/autogen/93/000000093-000000001.tsm
Dec 2 00:21:04 pi-server influxd[1321]: ts=2019-12-01T23:21:04.627864Z lvl=info msg=“Compacting file” log_id=0JSvpwlG000 engine=tsm1 tsm1_strategy=full tsm1_optimize=false trace_id=0JSx7XSl000 op_name=tsm1_compact_group tsm1_index=9 tsm1_file=/var/lib/influxdb/data/telegraf/autogen/93/000000094-000000001.tsm
Dec 2 00:21:04 pi-server influxd[1321]: ts=2019-12-01T23:21:04.628324Z lvl=info msg=“Aborted compaction” log_id=0JSvpwlG000 engine=tsm1 tsm1_strategy=full tsm1_optimize=false trace_id=0JSx7XSl000 op_name=tsm1_compact_group error=“compaction in progress: open /var/lib/influxdb/data/telegraf/autogen/93/000000094-000000002.tsm.tmp: file exists”
Contributor guide
Research direction
Start by reviewing the reported InfluxDB 1.7.9 TSM compaction log, especially the existing .tsm.tmp file error under /var/lib/influxdb/data/telegraf/autogen/93/. Investigate how repeated compaction handles that condition on Linux and Raspberry Pi; done means the cause is reproducible or explained and compaction no longer aborts in that case without data loss.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- linux, sqlite
- Domain
- databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100