apple / apple/foundationdb

Log more data distribution statistics

Open
#2,010 0 comments 2 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
16.7k
Forks
1.6k
Avg merge
1d 20h
Merged PRs (30d)
126

Description

There are a variety of questions regarding the behavior of data distribution that I've wanted answered but that have been difficult to determine due to a lack of metrics. Below are some various things I've wanted to know, without too much regard for the difficulty of adding a metric for each:

* How much data is being read by data distribution from each process
* Statistics about the number of shards (maybe bytes?) being moved. I think we already have some numbers that we could use to estimate this, but it seems like having counters that get incremented when relocations start and end would be easy enough and more accurate. It may also be nice to have counts by process for the sources and destinations.
* DD Queue information for each process
* A record of teams and machine teams that persists (e.g. by rolling the events)
* Some measurement of starvation at various DD priorities.

Contributor guide

Open the contributing guide

Research direction

The issue is a broad request for data-distribution metrics, covering read volume, shard movement, queue information, team history, and starvation. Start by reviewing the data distribution subsystem and its existing metric instrumentation, then determine which measurements have an agreed scope. Done would require a defined, implemented set of metrics with validation that they expose the requested behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
databases, distributed-systems, observability
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.