cockroachdb / cockroachdb/pebble

db: collect effective read-amp metric

Open
#5,015 0 comments 0 reactions 0 assignees View on GitHub
A-storage T-storage
Dominant language
Go
Stars
6k
Forks
584
Avg merge
16h 35m
Merged PRs (30d)
5

Description

We maintain a read amplification metric that we surface in CockroachDB as a timeseries metric. This metric is the worst-case read amplification in terms of the number of sstables consulted.

We should collect additional metrics that allow us to measure the effective read-amp actually observed by readers too. Optimizations like #2366 (or REMIX or DISCO papers) do not reduce the worst-case read amplification but can significantly reduce the number of read I/Os in practice.

Jira issue: PEBBLE-528

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by locating Pebble's existing worst-case read amplification metric and how it is surfaced to CockroachDB as a timeseries metric. Read the context in #2366 and compare the requested effective reader-observed read I/O measurements with the existing metric; done means additional useful metrics are collected and surfaced without replacing the worst-case measure.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
databases, observability, performance
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.