cockroachdb / cockroachdb/cockroach

kvserver: GetEngineCapacity is linear in number of sideloaded files

Open
#143,345 0 comments 0 reactions 0 assignees View on GitHub
branch-master C-bug T-kv
Dominant language
Go
Stars
32.5k
Forks
4.1k
PR merge metrics
PR metrics pending

Description

**Describe the problem**

Seen during scale testing [here](https://cockroachlabs.slack.com/archives/C08EHLCJH4G/p1742652420874129?thread_ts=1742612684.476649&cid=C08EHLCJH4G).

AddSST calls `GetEngineCapacity`. But `GetEngineCapacity` calls `pebble.Capacity` which has to recursively sum up the SST file size counts.

Image

https://github.com/cockroachdb/cockroach/blob/669cbd4c5a3e38a86bbb9c89ee80b725a0f9d8c3/pkg/storage/pebble.go#L1747-L1751

This doesn't seem ideal. It could be fixed at multiple levels:

- cache the result of `GetEngineCapacity` (how much can it change in a minute I guess?)
- make a separate method for free disk space, since that's what AddSST wants to know [1]: https://github.com/cockroachdb/cockroach/blob/669cbd4c5a3e38a86bbb9c89ee80b725a0f9d8c3/pkg/kv/kvserver/batcheval/cmd_add_sstable.go#L143-L157 [1]

[1]: here's pebble's `vfs.defaultFS.GetDiskUsage` method, which is ~cheap and does all we need already, though there are some bells and whistles to add if pebble is configured with a max size.

```go
func (defaultFS) GetDiskUsage(path string) (DiskUsage, error) {
stat := unix.Statfs_t{}
if err := unix.Statfs(path, &stat); err != nil {
return DiskUsage{}, err
}

freeBytes := uint64(stat.Bsize) * uint64(stat.Bfree)
availBytes := uint64(stat.Bsize) * uint64(stat.Bavail)
totalBytes := uint64(stat.Bsize) * uint64(stat.Blocks)
return DiskUsage{
AvailBytes: availBytes,
TotalBytes: totalBytes,
UsedBytes: totalBytes - freeBytes,
}, nil
}

```

Jira issue: CRDB-48766

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.