cockroachdb / cockroachdb/pebble

db: fine-grained profiling

Open
#3,957 1 comment 0 reactions 0 assignees View on GitHub
A-storage C-enhancement T-storage
Dominant language
Go
Stars
6k
Forks
584
Avg merge
16h 35m
Merged PRs (30d)
5

Description

This issue tracks implementing Pebble infrastructure for obtaining data on resource utilization (like CPU, disk bandwidth) broken down on various axes:
- CRDB key type (lock, txn, intent, table)
- read path / write path
- compression / decompression
- flushes / compaction (by target level)

Possibilities for tracked resources:
- CPU usage
- disk write bandwidth, iops
- disk read bandwidth, iops
- bloom filter negatives, false positives

The data should allow easily answering questions like "in this workload, how much CPU usage was caused by compactions of intent keys?"

We can implement this using pprof labels. For resources other than CPU we can maintain the statistics ourselves, as a map from label subsets. We could output them as pprof files or some other format.

This infrastructure can be behind a build flag to avoid adding overhead.

Jira issue: PEBBLE-260

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.