influxdata / influxdata/influxdb

Every 2-3 days "Aborted compaction"

Open
#16,136 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
Rust
Stars
31.7k
Forks
3.7k
Avg merge
13h 37m
Merged PRs (30d)
8

Description

__Steps to reproduce:__
I don't have any idea how to reproduce it.

__Actual behavior:__
My InfluxDB is running on Raspberry Pi. 2-10 Raspis are writing data into it. After 2-4 days I have this problem:

Aborted compaction

I switched from TSM to TSI. But the problem still exist. And now I lost new data (whole week).

Why I have so many problems? How can I solve it? I can't run SQL DB on the Raspberry Pi.

__Environment info:__

* System info: Linux pi-server 4.19.75-v7+ #1270 SMP Tue Sep 24 18:45:11 BST 2019 armv7l GNU/Linux
* InfluxDB version: 1.7.9
* Other relevant environment details: SQLlite

__Logs:__
Dec 2 00:21:03 pi-server influxd[1321]: ts=2019-12-01T23:21:03.628706Z lvl=info msg=“TSM compaction (end)” log_id=0JSvpwlG000 engine=tsm1 tsm1_strategy=full tsm1_optimize=false trace_id=0JSx7Pdl000 op_name=tsm1_compact_group op_event=end op_elapsed=1001.551ms
Dec 2 00:21:04 pi-server influxd[1321]: ts=2019-12-01T23:21:04.627120Z lvl=info msg=“TSM compaction (start)” log_id=0JSvpwlG000 engine=tsm1 tsm1_strategy=full tsm1_optimize=false trace_id=0JSx7XSl000 op_name=tsm1_compact_group op_event=start
Dec 2 00:21:04 pi-server influxd[1321]: ts=2019-12-01T23:21:04.627250Z lvl=info msg=“Beginning compaction” log_id=0JSvpwlG000 engine=tsm1 tsm1_strategy=full tsm1_optimize=false trace_id=0JSx7XSl000 op_name=tsm1_compact_group tsm1_files_n=10
Dec 2 00:21:04 pi-server influxd[1321]: ts=2019-12-01T23:21:04.627310Z lvl=info msg=“Compacting file” log_id=0JSvpwlG000 engine=tsm1 tsm1_strategy=full tsm1_optimize=false trace_id=0JSx7XSl000 op_name=tsm1_compact_group tsm1_index=0 tsm1_file=/var/lib/influxdb/data/telegraf/autogen/93/000000032-000000003.tsm
Dec 2 00:21:04 pi-server influxd[1321]: ts=2019-12-01T23:21:04.627374Z lvl=info msg=“Compacting file” log_id=0JSvpwlG000 engine=tsm1 tsm1_strategy=full tsm1_optimize=false trace_id=0JSx7XSl000 op_name=tsm1_compact_group tsm1_index=1 tsm1_file=/var/lib/influxdb/data/telegraf/autogen/93/000000064-000000003.tsm
Dec 2 00:21:04 pi-server influxd[1321]: ts=2019-12-01T23:21:04.627436Z lvl=info msg=“Compacting file” log_id=0JSvpwlG000 engine=tsm1 tsm1_strategy=full tsm1_optimize=false trace_id=0JSx7XSl000 op_name=tsm1_compact_group tsm1_index=2 tsm1_file=/var/lib/influxdb/data/telegraf/autogen/93/000000072-000000002.tsm
Dec 2 00:21:04 pi-server influxd[1321]: ts=2019-12-01T23:21:04.627497Z lvl=info msg=“Compacting file” log_id=0JSvpwlG000 engine=tsm1 tsm1_strategy=full tsm1_optimize=false trace_id=0JSx7XSl000 op_name=tsm1_compact_group tsm1_index=3 tsm1_file=/var/lib/influxdb/data/telegraf/autogen/93/000000081-000000002.tsm
Dec 2 00:21:04 pi-server influxd[1321]: ts=2019-12-01T23:21:04.627559Z lvl=info msg=“Compacting file” log_id=0JSvpwlG000 engine=tsm1 tsm1_strategy=full tsm1_optimize=false trace_id=0JSx7XSl000 op_name=tsm1_compact_group tsm1_index=4 tsm1_file=/var/lib/influxdb/data/telegraf/autogen/93/000000089-000000002.tsm
Dec 2 00:21:04 pi-server influxd[1321]: ts=2019-12-01T23:21:04.627620Z lvl=info msg=“Compacting file” log_id=0JSvpwlG000 engine=tsm1 tsm1_strategy=full tsm1_optimize=false trace_id=0JSx7XSl000 op_name=tsm1_compact_group tsm1_index=5 tsm1_file=/var/lib/influxdb/data/telegraf/autogen/93/000000090-000000001.tsm
Dec 2 00:21:04 pi-server influxd[1321]: ts=2019-12-01T23:21:04.627681Z lvl=info msg=“Compacting file” log_id=0JSvpwlG000 engine=tsm1 tsm1_strategy=full tsm1_optimize=false trace_id=0JSx7XSl000 op_name=tsm1_compact_group tsm1_index=6 tsm1_file=/var/lib/influxdb/data/telegraf/autogen/93/000000091-000000001.tsm
Dec 2 00:21:04 pi-server influxd[1321]: ts=2019-12-01T23:21:04.627742Z lvl=info msg=“Compacting file” log_id=0JSvpwlG000 engine=tsm1 tsm1_strategy=full tsm1_optimize=false trace_id=0JSx7XSl000 op_name=tsm1_compact_group tsm1_index=7 tsm1_file=/var/lib/influxdb/data/telegraf/autogen/93/000000092-000000001.tsm
Dec 2 00:21:04 pi-server influxd[1321]: ts=2019-12-01T23:21:04.627803Z lvl=info msg=“Compacting file” log_id=0JSvpwlG000 engine=tsm1 tsm1_strategy=full tsm1_optimize=false trace_id=0JSx7XSl000 op_name=tsm1_compact_group tsm1_index=8 tsm1_file=/var/lib/influxdb/data/telegraf/autogen/93/000000093-000000001.tsm
Dec 2 00:21:04 pi-server influxd[1321]: ts=2019-12-01T23:21:04.627864Z lvl=info msg=“Compacting file” log_id=0JSvpwlG000 engine=tsm1 tsm1_strategy=full tsm1_optimize=false trace_id=0JSx7XSl000 op_name=tsm1_compact_group tsm1_index=9 tsm1_file=/var/lib/influxdb/data/telegraf/autogen/93/000000094-000000001.tsm
Dec 2 00:21:04 pi-server influxd[1321]: ts=2019-12-01T23:21:04.628324Z lvl=info msg=“Aborted compaction” log_id=0JSvpwlG000 engine=tsm1 tsm1_strategy=full tsm1_optimize=false trace_id=0JSx7XSl000 op_name=tsm1_compact_group error=“compaction in progress: open /var/lib/influxdb/data/telegraf/autogen/93/000000094-000000002.tsm.tmp: file exists”

Contributor guide

Open the contributing guide

Research direction

Start by reviewing the reported InfluxDB 1.7.9 TSM compaction log, especially the existing .tsm.tmp file error under /var/lib/influxdb/data/telegraf/autogen/93/. Investigate how repeated compaction handles that condition on Linux and Raspberry Pi; done means the cause is reproducible or explained and compaction no longer aborts in that case without data loss.

Written by the indexing model from the issue text.

Assessment

Tech stack
linux, sqlite
Domain
databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.