cockroachdb / cockroachdb/cockroach

opt: locality optimized lookup join doesn't get planned when expected

Open
#100,896 1 comment 0 reactions 0 assignees View on GitHub
A-multiregion A-sql-optimizer C-bug O-qa T-sql-queries
Dominant language
Go
Stars
32.5k
Forks
4.1k
PR merge metrics
PR metrics pending

Description

Run on master with
```
cockroach demo movr --global --nodes 9 --multitenant=false --insecure
```
and then convert a couple of tables into multi-region
```sql
ALTER DATABASE movr PRIMARY REGION "us-east1";
ALTER DATABASE movr ADD REGION "europe-west1";
ALTER DATABASE movr ADD REGION "us-west1";

ALTER TABLE user_promo_codes ADD COLUMN region crdb_internal_region AS (
CASE WHEN city = 'amsterdam' THEN 'europe-west1'
WHEN city = 'paris' THEN 'europe-west1'
WHEN city = 'rome' THEN 'europe-west1'
WHEN city = 'new york' THEN 'us-east1'
WHEN city = 'boston' THEN 'us-east1'
WHEN city = 'washington dc' THEN 'us-east1'
WHEN city = 'san francisco' THEN 'us-west1'
WHEN city = 'seattle' THEN 'us-west1'
WHEN city = 'los angeles' THEN 'us-west1'
ELSE 'us-east1'
END
) STORED;
ALTER TABLE user_promo_codes ALTER COLUMN region SET NOT NULL;
ALTER TABLE user_promo_codes SET LOCALITY REGIONAL BY ROW AS "region";

ALTER TABLE users ADD COLUMN region crdb_internal_region AS (
CASE WHEN city = 'amsterdam' THEN 'europe-west1'
WHEN city = 'paris' THEN 'europe-west1'
WHEN city = 'rome' THEN 'europe-west1'
WHEN city = 'new york' THEN 'us-east1'
WHEN city = 'boston' THEN 'us-east1'
WHEN city = 'washington dc' THEN 'us-east1'
WHEN city = 'san francisco' THEN 'us-west1'
WHEN city = 'seattle' THEN 'us-west1'
WHEN city = 'los angeles' THEN 'us-west1'
ELSE 'us-east1'
END
) STORED;
ALTER TABLE users ALTER COLUMN region SET NOT NULL;
ALTER TABLE users SET LOCALITY REGIONAL BY ROW AS "region";
```
I expect that the query below would get locality-optimized lookup join (since it's effectively doing the same thing as an example from [here](https://github.com/cockroachdb/cockroach/pull/93377#issue-1488270428)) but it doesn't:
```sql
EXPLAIN (OPT) SELECT * FROM user_promo_codes c INNER LOOKUP JOIN users u ON c.city = u.city AND c.user_id = u.id LIMIT 3;
```
```
inner-join (lookup users [as=u])
├── flags: force lookup join (into right side)
├── lookup columns are key
├── project
│ ├── locality-optimized-search
│ │ ├── scan user_promo_codes [as=c]
│ │ │ ├── constraint: /23/18/19/20: [/'us-east1' - /'us-east1']
│ │ │ └── limit: 3
│ │ └── scan user_promo_codes [as=c]
│ │ ├── constraint: /31/26/27/28
│ │ │ ├── [/'europe-west1' - /'europe-west1']
│ │ │ └── [/'us-west1' - /'us-west1']
│ │ └── limit: 3
│ └── projections
│ └── CASE WHEN c.city = 'amsterdam' THEN 'europe-west1' WHEN c.city = 'paris' THEN 'europe-west1' WHEN c.city = 'rome' THEN 'europe-west1' WHEN c.city = 'new york' THEN 'us-east1' WHEN c.city = 'boston' THEN 'us-east1' WHEN c.city = 'washington dc' THEN 'us-east1' WHEN c.city = 'san francisco' THEN 'us-west1' WHEN c.city = 'seattle' THEN 'us-west1' WHEN c.city = 'los angeles' THEN 'us-west1' ELSE 'us-east1' END
└── filters (true)
```
Perhaps it's the projection that is preventing that, or perhaps I'm misunderstanding something.

Note that initially I tried `CASE` statements for the new manually added columns without the `ELSE` part, in which case the `region` column could be NULL, but adding the `ELSE` didn't change anything (in theory, this shouldn't matter since we have NOT NULL constraint on the column).

cc @msirek

Jira issue: CRDB-26678

Contributor guide

Open the contributing guide

Research direction

Reproduce the issue with the cockroach demo movr command and the supplied multi-region ALTER statements, then run the EXPLAIN (OPT) lookup-join query. Start by comparing the observed locality-optimized-search plan with the expected locality-optimized lookup join. Done means identifying and correcting why this plan is not selected, with the query producing the expected optimized plan.

Written by the indexing model from the issue text.

Assessment

Tech stack
go, sql
Domain
databases, distributed-systems
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.