apple / apple/foundationdb

Metrics on throttled transactions

Open
#5,063 3 comments 0 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
16.7k
Forks
1.6k
Avg merge
1d 20h
Merged PRs (30d)
126

Description

According to 6.3.15, we have metrics on throttled transactions at two places `ProxyMetrics::TxnThrottled` and `TransactionMetrics::ReadVersionsThrottled`.

Looked at that code, those KPIs seem very different from what they sound like. If I can summarize in FDB client terms

- `ProxyMetrics::TxnThrottled` counts the transactions that are throttled before adding to queues. It seems like this can happen only for batch priority. Although, not all batch priority transactions throttled would be counted under this as they can throttled in later phases
- `TransactionMetrocs::ReadVersionsThrottled` counts the transactions that are throttled after they queued in proxy Qs. But, it seems this counts only transactions that have tags.

These two KPIs seem to count complete disjoint set of transactions that are throttled. But, if we add them would that be complete set of transactions that are throttled? Seems like we still miss queued transactions that are throttled without tags. Unless we have a default tag added to all transactions.

Feels like this is not the desired behavior. Ideally we should have `ProxyMetrics` showing all transactions throttled on that proxy and `TransactionMetrics` showing all throttled transactions started from that client.

Contributor guide

Open the contributing guide

Research direction

Start by tracing updates to ProxyMetrics::TxnThrottled and TransactionMetrics::ReadVersionsThrottled through the proxy queue and transaction throttling paths, paying attention to batch priority and tagged versus untagged transactions. Compare the counted sets with the intended client- and proxy-level definitions, then document or implement a consistent accounting model and verify that all throttled transactions are represented.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
databases, distributed-systems, observability
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.