influxdata / influxdata/influxdb

vmware snapshots break influxdb container/host

Open
#22,276 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Rust
Stars
31.7k
Forks
3.7k
Avg merge
13h 37m
Merged PRs (30d)
8

Description

__Steps to reproduce:__
List the minimal actions needed to reproduce the behavior.

1. snapshot/backup VM where influxdb is running
2. influxdb remains running, POST/GET queries respond with 20X responses
3. No data is returned in queries
4. Entire host requires a reboot, not just the container

__Expected behavior:__
VM snapshots/backups don't impact influxdb.

__Actual behavior:__
Influxdb appears healthy, but doesn't return any data, throw any errors and responds to POST/writes with 200~ status codes, however this data is lost. Until the VM is restarted, influxdb returns to normal operation.

The issue is reproducable with Veeam backups, creating VM snapshots with CBT, backup = broken influx.

__Environment info:__

* System info: Linux 4.18.0-305.12.1.el8_4.x86_64 x86_64
* InfluxDB version: InfluxDB v1.8.6 (git: 1.8 v1.8.6)
* Other relevant environment details: Rocky Linux 8.4, Kernel 4.18, Docker 20.10.8, vmware pv local disk/NVMe SSD. 2vCPU/4GB RAM.

__Config:__
No custom config

__Logs:__
No relevant logs.

__Performance:__

Contributor guide

Open the contributing guide

Research direction

No source files or tests are identified in the report. Start by reproducing the Veeam or VMware snapshot scenario with InfluxDB v1.8.6 in the stated Docker and Linux environment, then compare query and write behavior before and after the snapshot. Done means the snapshot no longer causes silent data loss, or the failure is reported clearly.

Written by the indexing model from the issue text.

Assessment

Tech stack
docker, linux
Domain
database, infrastructure
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.