cockroachdb / cockroachdb/cockroach
sql: generic query plans for region-by-row (RBR) multi-region tables
- Dominant language
- Go
- Stars
- 32.5k
- Forks
- 4.1k
- PR merge metrics
- PR metrics pending
Description
There are a number of things preventing generic query plans from being used for queries on `REGIONAL BY ROW` tables.
### A.
This restriction prevents generic query plans from being used on multi-region tables: https://github.com/cockroachdb/cockroach/blob/79a6fc810e811805d66d58c0d565d5d253e70539/pkg/sql/plan_opt.go#L404-L413
We should find a way to lift this restriction.
### B.
If we do lift the restriction in A., we'll have to ensure that `enforce_home_region` works as expected with generic query plans. `encforce_home_region` currently uses execbuilder-time checks to verify that queries are not accessing rows that are not in the gateway region. With generic query plans, which use parameterized lookup-joins, the distribution of a plan may not be fully understood at execbuilder-time. I expect this will cause some generic query plans to incorrectly fail because it cannot be proven that the query only accesses rows in the gateway region.
### C.
Also, the logic mention in A. is suspect. Because it uses the eval context of the session—which seems to maintain state between txns—once a session executes a query that access a regional-by-row, all future queries will not be able to use generic query plans. This seems like a bug.
Here's an example in a logictest showing that a generic query plan can be used in an MR database:
```
# LogicTest: multiregion-9node-3region-3azs !metamorphic-batch-sizes
statement ok
CREATE DATABASE multi_region_test_db PRIMARY REGION "ca-central-1" REGIONS "ap-southeast-2", "us-east-1" SURVIVE ZONE FAILURE
statement ok
USE multi_region_test_db
sleep 5s
statement ok
CREATE TABLE t (
k INT PRIMARY KEY,
v STRING
)
statement ok
SET plan_cache_mode = force_generic_plan
statement ok
PREPARE P AS SELECT * FROM t WHERE k = $1 AND v = $2
# A generic query plan is NOT used!
query T match(plan\stype)
EXPLAIN ANALYZE EXECUTE p(5, 'five')
----
plan type: generic, re-optimized
```
But generic query plans are not used once a RBR table is accessed (and I believe setting `enforce_home_region` is required too):
```
# LogicTest: multiregion-9node-3region-3azs !metamorphic-batch-sizes
statement ok
CREATE DATABASE multi_region_test_db PRIMARY REGION "ca-central-1" REGIONS "ap-southeast-2", "us-east-1" SURVIVE ZONE FAILURE
statement ok
USE multi_region_test_db
statement ok
CREATE TABLE table_regional_by_row (
k INT PRIMARY KEY,
V STRING
) LOCALITY REGIONAL BY ROW
sleep 5s
statement ok
INSERT INTO table_regional_by_row (crdb_region, k, v) VALUES ('ap-southeast-2', 5, 'five'), ('us-east-1', 6, 'six')
statement ok
SET enforce_home_region = true
query T
SELECT v FROM table_regional_by_row@{AVOID_FULL_SCAN} WHERE crdb_region = 'ap-southeast-2' AND k = 5
----
five
statement ok
SET enforce_home_region = false
statement ok
CREATE TABLE t (
k INT PRIMARY KEY,
v STRING
)
statement ok
SET plan_cache_mode = force_generic_plan
statement ok
PREPARE P AS SELECT * FROM t WHERE k = $1 AND v = $2
# A generic query plan is NOT used!
query T match(plan\stype)
EXPLAIN ANALYZE EXECUTE p(5, 'five')
----
plan type: custom
```
Jira issue: CRDB-51535
Contributor guide
Assessment
This issue has not been assessed yet.