influxdata / influxdata/influxdb
2.0 online backup performance
- Dominant language
- Rust
- Stars
- 31.7k
- Forks
- 3.7k
- Avg merge
- 13h 37m
- Merged PRs (30d)
- 8
Description
In a busy InfluxDB 1.x instance, backups occasionally timeout due to constant snapshotting. https://github.com/influxdata/plutonium/issues/2997 We should check for this in 2.x.
Need to perform backup performance tests. Backup should be fast even when lots of compactions are active, without creating a compaction backlog.
A good benchmark would be:
- 10GB of data on disk
- cardinality about 5 million
- write at a fast rate (TBD) points per second
- observe plenty of snapshot and compaction activity
- do a backup
The most critical metric to check should be the latency of POST to /api/v2/backup. This creates TSM file snapshots, which are hard links. In theory this should be fast, but we have to acquire an exclusive lock on the entire TSM fileset in order to do it correctly.
Contributor guide
Assessment
This issue has not been assessed yet.