hashicorp / hashicorp/consul

Benchmark compression on Raft Snapshots to reduce disk IO

Open
#5,360 0 comments 0 reactions 0 assignees View on GitHub
needs-investigation theme/performance type/enhancement
Dominant language
Go
Stars
30.1k
Forks
4.6k
Avg merge
2d 6h
Merged PRs (30d)
43

Description

Currently raft-initiated snapshots are stored on disk as plain length-prefixed msgpack serialised records.

We hypothesise that under heavy write load with large data sizes, the disk IO of writing out large snapshots to disk regularly causes raft throughput to tank since the snapshot is written to the same device as the raft log.

Given that, it _might_ be an easy-ish performance win for IO bound write workloads if we allow optional compression. GZIP might even be OK given the async nature of the snapshot process but faster compression algorithms like Snappy or LZ4 might also reduce IO considerably with lower CPU cost.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.