cockroachdb / cockroachdb/cockroach
kvserver: GetEngineCapacity is linear in number of sideloaded files
- Dominant language
- Go
- Stars
- 32.5k
- Forks
- 4.1k
- PR merge metrics
- PR metrics pending
Description
**Describe the problem**
Seen during scale testing [here](https://cockroachlabs.slack.com/archives/C08EHLCJH4G/p1742652420874129?thread_ts=1742612684.476649&cid=C08EHLCJH4G).
AddSST calls `GetEngineCapacity`. But `GetEngineCapacity` calls `pebble.Capacity` which has to recursively sum up the SST file size counts.
https://github.com/cockroachdb/cockroach/blob/669cbd4c5a3e38a86bbb9c89ee80b725a0f9d8c3/pkg/storage/pebble.go#L1747-L1751
This doesn't seem ideal. It could be fixed at multiple levels:
- cache the result of `GetEngineCapacity` (how much can it change in a minute I guess?)
- make a separate method for free disk space, since that's what AddSST wants to know [1]: https://github.com/cockroachdb/cockroach/blob/669cbd4c5a3e38a86bbb9c89ee80b725a0f9d8c3/pkg/kv/kvserver/batcheval/cmd_add_sstable.go#L143-L157 [1]
[1]: here's pebble's `vfs.defaultFS.GetDiskUsage` method, which is ~cheap and does all we need already, though there are some bells and whistles to add if pebble is configured with a max size.
```go
func (defaultFS) GetDiskUsage(path string) (DiskUsage, error) {
stat := unix.Statfs_t{}
if err := unix.Statfs(path, &stat); err != nil {
return DiskUsage{}, err
}
freeBytes := uint64(stat.Bsize) * uint64(stat.Bfree)
availBytes := uint64(stat.Bsize) * uint64(stat.Bavail)
totalBytes := uint64(stat.Bsize) * uint64(stat.Blocks)
return DiskUsage{
AvailBytes: availBytes,
TotalBytes: totalBytes,
UsedBytes: totalBytes - freeBytes,
}, nil
}
```
Jira issue: CRDB-48766
Contributor guide
Assessment
This issue has not been assessed yet.