cockroachdb / cockroachdb/cockroach

kvcoord: DistSender circuit breaker cancellation won't work on local requests

Open
#121,210 1 comment 0 reactions 0 assignees View on GitHub
A-kv-client C-bug T-kv
Dominant language
Go
Stars
32.5k
Forks
4.1k
PR merge metrics
PR metrics pending

Description

The DistSender circuit breakers will cancel in-flight requests when the breaker trips, such that they can be retried on a different replica instead of getting stuck. However, context cancellation may be ineffective on requests that are processed locally, since it is not respected by disk IO, syscalls, mutex acquisition, etc.

We should see if there is a way to fix this -- a straightforward but likely too expensive option is to spawn a separate goroutine for the request processing, where the client selects on a result channel along with the context channel. Another option is to use e.g. async IO, or lobby the Go team to add context support for various APIs such as IO and mutexes.

We likely can't do anything about this for 24.1, but I'm marking it as a GA blocker for visibility.

Jira issue: CRDB-37140

Epic CRDB-39897

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.