cockroachdb / cockroachdb/cockroach

sql: simplify going from txn ID to statements within that txn

Open
#149,942 3 comments 0 reactions 1 assignee Claimed by @mw5h View on GitHub
A-sql-observability C-enhancement O-support P-3 T-sql-queries
Dominant language
Go
Stars
32.5k
Forks
4.1k
PR merge metrics
PR metrics pending

Description

Whenever we observe contention, we usually see the transaction ID that our stmt contended with (e.g. this can be found in the trace from the stmt bundle). However, it's currently close to impossible to go from that transaction ID to some other information, especially what we persist in system tables. This is the case since we use "transaction fingerprint ID" (which is a hash of all stmts within a txn) when writing into `system.statement_statistics` and `system.transaction_statistics` tables.

We've built the txn ID resolution cache in order to power `crdb_internal.{node,cluster}_contention_events` virtual tables. We should expose this cache as another virtual table so that we could do the "txn ID -> txn fingerprint ID resolution" (which is currently only possible via gRPC invocation) as long the cache hasn't rotated over the txn we're interested in. Some other ideas on further improvements:
- add a virtual index on this virtual table so that only txns that were committed within the specified time window would be included
- consider "touching" the entry in the cache - to keep it in the cache for longer - whenever that cache entry is looked up for a contention event. The thinking here is that if we resolved a txn ID but there was no contention reported with that txn, it's less likely to be useful
- estimate how many entries we currently hold in the cache and, perhaps, increase the cache size
- consider including some of the contents of the new virtual table into the debug.zip. Perhaps the cache could also keep track of the number of times a particular txn was looked for a contention event, and we'd only dump cache entries that were looked up at least once.

On a related theme, we could consider augmenting the stmt bundle building code so that when we finalize the bundle, we'd look over the trace, and if we see any txns on which we waited there, we'd wait for a configurable amount of time until the txn ID cache has resolved interesting to use txns. Then we'd be able to include transaction fingerprint IDs into the bundle as a separate file.

See [this](https://cockroachlabs.slack.com/archives/C0941RFDJLV/p1752174640842189) slack thread for the current struggle.

Jira issue: CRDB-52425

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.