cockroachdb / cockroachdb/cockroach
sql,storage: avoid retrieving value for column families not needed by MVCC scan
- Dominant language
- Go
- Stars
- 32.5k
- Forks
- 4.1k
- PR merge metrics
- PR metrics pending
Description
With the upcoming Pebble value separation, we should start recommending that rarely accessed giant columns be placed in a separate column family. See https://cockroachlabs.slack.com/archives/CHVV403F0/p1756145796696309 for a possible customer use case.
Column families are interleaved in CockroachDB, in that for an index with three column families, 0, 1, 2, the index key will have the column family encoded at the end. Say a SQL query needs to scan keys [1-100] for column family 1. We currently do a MVCC scan with the span [/table/index/1/cf1, /table/index/100/cf1]. This causes us to retrieve the values of keys [1-99] for cf0 and cf1 too, which is wasteful for multiple reasons (a) these values could be in separate files in Pebble if they are large, and costly to retrieve, (b) we will serialize and send them back to SQL (in the absence of projection pushdown). Instead, if SQL plumbs a column-family-ID filter into the scan, the code in `pebbleMVCCScanner` can decode the family ID (using `DecodeFamilyKey`) and if it doesn't match the filter ignore it.
This is complementary to the (not enabled, though hopefully enabled in the future) implementation of projection pushdown via `cFetcherWrapper`, in that the projection happens after the value is retrieved, while the above can avoid retrieving the value.
See https://cockroachlabs.slack.com/archives/C01RX2G8LT1/p1756230506958379 for internal discussion.
@yuzefovich
Jira issue: CRDB-53908
Contributor guide
Assessment
This issue has not been assessed yet.