apple / apple/foundationdb

Add the ability to track internally generated and externally generated load separately

Open
#1,692 0 comments 4 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
16.7k
Forks
1.6k
Avg merge
1d 20h
Merged PRs (30d)
126

Description

There are a variety of sources of internal load on a cluster, and this can often cloud discussions with cluster users about their workloads. It also makes it harder to determine the effects of various internal processes.

It would be useful if we could have some way to distinguish the load generated by internal activities (e.g. data distribution, backup/DR, consistency check, etc) from those generated by clients of the database. Perhaps even better from a tooling standpoint would be a more general way to distinguish internal transactions by category (so we could, for example, understand the effects of DD by itself).

This may require us to mark each request with additional information about the source of the traffic so that the process handling the request can increment the appropriate counters. In that case, we'll have to balance the functionality with the cost of sending the required information.

Contributor guide

Open the contributing guide

Research direction

No files, tests, or entry points are named. Start by locating request handling and accounting for data distribution, backup/DR, consistency checks, and client traffic; done would mean these sources can be distinguished, ideally by internal category, while keeping the added request information cost acceptable.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
databases, distributed-systems
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.