apple / apple/foundationdb

Add write/read bandwidth info of a shard to special key space metrics module

Open
#7,490 0 comments 0 reactions 0 assignees View on GitHub
data distribution
Dominant language
C++
Stars
16.7k
Forks
1.6k
Avg merge
1d 20h
Merged PRs (30d)
126

Description

Now in special key space, through
```
>>> for k, v in db.get_range_startswith('\xff\xff/metrics/data_distribution_stats/', limit=3):
... print(k, v)
...
('\xff\xff/metrics/data_distribution_stats/', '{"shard_bytes":3828000}')
('\xff\xff/metrics/data_distribution_stats/mako00079', '{"shard_bytes":2013000}')
('\xff\xff/metrics/data_distribution_stats/mako00126', '{"shard_bytes":3201000}')
```
We can just get the shard estimated size. Since we have some shard rebalance and shard merge/split action also relies on write/read bandwidth, it's helpful to add these info for better DD observability.

Contributor guide

Open the contributing guide

Research direction

Start in the special key space metrics module and inspect the existing data_distribution_stats entries, including the shard_bytes values shown in the issue. Determine how write and read bandwidth should be represented alongside shard size, and consider the rebalance and shard merge/split consumers as the completion criteria for the observability data.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
databases, distributed-systems, observability
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.