cockroachdb / cockroachdb/cockroach

storage: tune reserved space based on filesystem type

Open
#93,178 2 comments 0 reactions 0 assignees View on GitHub
A-storage C-enhancement T-storage
Dominant language
Go
Stars
32.5k
Forks
4.1k
PR merge metrics
PR metrics pending

Description

Different filesystems should have different reserved capacity. Modern versions of ext[2-4][1]reserve 5% of blocks as the high watermark for available disk space (see `-m`). ZFS should use ~82% (value from personal experience, mostly due to the correlation between zpool utilization, disk fragmentation, and work required to find a free block to perform a write).

> looking at a roachtest out-of-disk failure, we noticed that the node crashed with `ENOSPC` at 95%. This is because ext4 reserves 5% space for the root user by default. However, this interacts poorly with some of our heuristics that try to halt writes at 95% to prevent running out of disk (e.g. AddSSTable ingestion and rebalancing), because at 95% we're already out of disk. This reserved space also serves a similar purpose to our ballast files, in that the root user can reallocate it to CockroachDB in an emergency.

[1] [mkfs.ext4](https://linux.die.net/man/8/mkfs.ext4)

Jira issue: CRDB-22206

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.