influxdata / influxdata/influxdata-docker

Influxdb Backup is not working properly in influxdb OSS V2.1.1

Open
#604 6 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

influxdb
Dominant language
Shell
Stars
364
Forks
255
Avg merge
3h 18m
Merged PRs (30d)
7

Description

## About the bug

**Steps to reproduce:**
List the minimal actions needed to reproduce the behavior.

1. just perform the backup cli command `influx backup -t {$rootToken} /var/backups/backup_$(date '+%Y-%m-%d_%H-%M')`
2. then check the backup log and files, data are incomplete

**Expected behavior:**

influxdb take a full backup of all the data stored in it, in my case it should be around 1.5-2.0 GB

**Actual behavior:**
data in the backup files are only 1.3 Mb, WARN: Shard xxxx removed during backup shown frequently in the log

**Visual Proof:**

![2](https://user-images.githubusercontent.com/102961908/161531150-f87f5508-3923-4e1f-98b3-dd124c430b91.png)
![1](https://user-images.githubusercontent.com/102961908/161531153-f4caf141-d140-4d5f-a7fe-1b60b1c34bbc.png)

data in influxdb seem to be ok, just the backup is buggy.
A downgrade back to v2.0 caused errors during the initialization of the container application.

## About your environment

**Environment info:**

influxdb OSS v2.1.1, installation by container

Linux 5.10.60-qnap x86_64
InfluxDB 2.1.1 (git: 657e1839de) build_date: 2021-11-09T03:03:48Z

**Config:**

```
version: '3.5'
networks:
influxdbnw:
name: influxdb_network
services:
influxdb:
image: 'influxdb:2.1.1'
restart: always
container_name: influxdb
hostname: 'influxdb'
volumes:
#- ./data:/root/.influxdbv2
- .\conf\influx\data:/var/lib/influxdb2
- .\conf\influx\config:/etc/influxdb2
- .\conf\influx\scripts:/docker-entrypoint-initdb.d
- '{§backup path}:/var/backups'
- ./config/influx-configs:/etc/influxdb2/influx-configs'
ports:
- '8086:8086'
networks:
- influxdbnw
environment:
- DOCKER_INFLUXDB_INIT_MODE=setup
- DOCKER_INFLUXDB_INIT_USERNAME=admin
- DOCKER_INFLUXDB_INIT_PASSWORD=password
- DOCKER_INFLUXDB_INIT_ORG=org
- DOCKER_INFLUXDB_INIT_BUCKET=initBucket
- DOCKER_INFLUXDB_INIT_ADMIN_TOKEN=token
- TZ=Europe/Berlin
restart: always
volumes:
data:
config:
scripts:

```

**Logs:**

see the screenshots above

**Performance:**
Generate profiles with the following commands for bugs related to performance, locking, out of memory (OOM), etc.

```sh
# Commands should be run when the bug is actively.
# Note: This command will run for at least 30 seconds.
curl -o profiles.tar.gz "http://localhost:8086/debug/pprof/all?cpu=true"
curl -o vars.txt "http://localhost:8086/debug/vars"
iostat -xd 1 30 > iostat.txt
# Attach the `profiles.tar.gz`, `vars.txt`, and `iostat.txt` output files.
```

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the `influx backup -t {$rootToken} /var/backups/backup_$(date '+%Y-%m-%d_%H-%M')` command and the provided Docker configuration for InfluxDB OSS 2.1.1. Compare the backup files and logs, focusing on repeated “Shard xxxx removed during backup” warnings. Done means a backup contains the expected complete data rather than only 1.3 MB.

Written by the indexing model from the issue text.

Assessment

Tech stack
docker, shell
Domain
databases, devops
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.