cockroachdb / cockroachdb/cockroach

kv: uniquely identify transaction deadlock errors

Open
#116,482 0 comments 0 reactions 0 assignees View on GitHub
A-kv-observability A-kv-transactions C-enhancement T-kv
Dominant language
Go
Stars
32.5k
Forks
4.1k
PR merge metrics
PR metrics pending

Description

Transaction deadlocks occur when two or more transactions acquire locks in an order that creates a cycle. In such cases, one of the transactions [must be aborted](https://github.com/cockroachdb/cockroach/blob/830dad6b1da950bc47e05c1116c7f7c61994bbca/pkg/kv/kvserver/txnwait/queue.go#L766) in order to break the deadlock. The aborted transaction will then notice that it has been aborted and return an aborted error.

Deadlock detection is one of the responsibilities of the `txnwait.Queue`. The algorithm is described here:

https://github.com/cockroachdb/cockroach/blob/830dad6b1da950bc47e05c1116c7f7c61994bbca/pkg/kv/kvserver/concurrency/concurrency_control.go#L881

Currently, the transaction who gets aborted by deadlock detection finds out through one of a few ways. Sometimes, it finds out [while waiting in the txnwait queue](https://github.com/cockroachdb/cockroach/blob/830dad6b1da950bc47e05c1116c7f7c61994bbca/pkg/kv/kvserver/txnwait/queue.go#L721), returning a `TransactionAbortedError(ABORT_REASON_PUSHER_ABORTED)`. Sometimes, its heartbeat notices the aborted txn record, leading to a `TransactionAbortedError(ABORT_REASON_ABORTED_RECORD_FOUND)`. It may also be possible that it ends up throwing a `TransactionAbortedError(ABORT_REASON_CLIENT_REJECT)` error.

This is unfortunate, as it makes the error difficult to explain. Ideally, there would be a single error code/reason that corresponds to a transaction deadlock abort. This would make the condition easier to document and allow for https://github.com/cockroachdb/cockroach/issues/116481.

To make an improvement here, it might be easiest to invert the deadlock detection resolution. Instead of having the winner abort the loser, the loser could abort itself and then return a new type of error.

Jira issue: CRDB-34640

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.