cockroachdb / cockroachdb/cockroach

sql,storage: avoid retrieving value for column families not needed by MVCC scan

Open
#152,541 2 comments 0 reactions 0 assignees View on GitHub
A-storage C-performance T-storage
Dominant language
Go
Stars
32.5k
Forks
4.1k
PR merge metrics
PR metrics pending

Description

With the upcoming Pebble value separation, we should start recommending that rarely accessed giant columns be placed in a separate column family. See https://cockroachlabs.slack.com/archives/CHVV403F0/p1756145796696309 for a possible customer use case.

Column families are interleaved in CockroachDB, in that for an index with three column families, 0, 1, 2, the index key will have the column family encoded at the end. Say a SQL query needs to scan keys [1-100] for column family 1. We currently do a MVCC scan with the span [/table/index/1/cf1, /table/index/100/cf1]. This causes us to retrieve the values of keys [1-99] for cf0 and cf1 too, which is wasteful for multiple reasons (a) these values could be in separate files in Pebble if they are large, and costly to retrieve, (b) we will serialize and send them back to SQL (in the absence of projection pushdown). Instead, if SQL plumbs a column-family-ID filter into the scan, the code in `pebbleMVCCScanner` can decode the family ID (using `DecodeFamilyKey`) and if it doesn't match the filter ignore it.

This is complementary to the (not enabled, though hopefully enabled in the future) implementation of projection pushdown via `cFetcherWrapper`, in that the projection happens after the value is retrieved, while the above can avoid retrieving the value.

See https://cockroachlabs.slack.com/archives/C01RX2G8LT1/p1756230506958379 for internal discussion.
@yuzefovich

Jira issue: CRDB-53908

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.