cockroachdb / cockroachdb/cockroach

roachtest: `failover` tests should track outage duration, not pMax

Open
#133,361 0 comments 0 reactions 0 assignees View on GitHub
A-kv A-testing C-enhancement T-kv
Dominant language
Go
Stars
32.5k
Forks
4.1k
PR merge metrics
PR metrics pending

Description

Currently, the `failover` test suite measures an outage by the pMax of any request experienced during the outage. This means that it cannot differentiate between an outage that causes some requests to hit timeouts for a short period of time, and an outage that causes some requests to continue to hit timeouts for a longer period of time. As a result, we have had to make changes like https://github.com/cockroachdb/cockroach/pull/133214.

This is a non-standard way to define outage which does not map to how our customers think about availability. We should switch the test to set a statement timeout and then measure the period where timeouts are hit.

Jira issue: CRDB-43556

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.