influxdata / influxdata/influxdb

2.0 online backup performance

Open
#15,969 0 comments 0 reactions 1 assignee Claimed by @jacobmarble View on GitHub
area/2.x area/performance
Dominant language
Rust
Stars
31.7k
Forks
3.7k
Avg merge
13h 37m
Merged PRs (30d)
8

Description

In a busy InfluxDB 1.x instance, backups occasionally timeout due to constant snapshotting. https://github.com/influxdata/plutonium/issues/2997 We should check for this in 2.x.

Need to perform backup performance tests. Backup should be fast even when lots of compactions are active, without creating a compaction backlog.

A good benchmark would be:
- 10GB of data on disk
- cardinality about 5 million
- write at a fast rate (TBD) points per second
- observe plenty of snapshot and compaction activity
- do a backup

The most critical metric to check should be the latency of POST to /api/v2/backup. This creates TSM file snapshots, which are hard links. In theory this should be fast, but we have to acquire an exclusive lock on the entire TSM fileset in order to do it correctly.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.