cockroachdb / cockroachdb/cockroach
opt: locality optimized lookup join doesn't get planned when expected
- Dominant language
- Go
- Stars
- 32.5k
- Forks
- 4.1k
- PR merge metrics
- PR metrics pending
Description
Run on master with
```
cockroach demo movr --global --nodes 9 --multitenant=false --insecure
```
and then convert a couple of tables into multi-region
```sql
ALTER DATABASE movr PRIMARY REGION "us-east1";
ALTER DATABASE movr ADD REGION "europe-west1";
ALTER DATABASE movr ADD REGION "us-west1";
ALTER TABLE user_promo_codes ADD COLUMN region crdb_internal_region AS (
CASE WHEN city = 'amsterdam' THEN 'europe-west1'
WHEN city = 'paris' THEN 'europe-west1'
WHEN city = 'rome' THEN 'europe-west1'
WHEN city = 'new york' THEN 'us-east1'
WHEN city = 'boston' THEN 'us-east1'
WHEN city = 'washington dc' THEN 'us-east1'
WHEN city = 'san francisco' THEN 'us-west1'
WHEN city = 'seattle' THEN 'us-west1'
WHEN city = 'los angeles' THEN 'us-west1'
ELSE 'us-east1'
END
) STORED;
ALTER TABLE user_promo_codes ALTER COLUMN region SET NOT NULL;
ALTER TABLE user_promo_codes SET LOCALITY REGIONAL BY ROW AS "region";
ALTER TABLE users ADD COLUMN region crdb_internal_region AS (
CASE WHEN city = 'amsterdam' THEN 'europe-west1'
WHEN city = 'paris' THEN 'europe-west1'
WHEN city = 'rome' THEN 'europe-west1'
WHEN city = 'new york' THEN 'us-east1'
WHEN city = 'boston' THEN 'us-east1'
WHEN city = 'washington dc' THEN 'us-east1'
WHEN city = 'san francisco' THEN 'us-west1'
WHEN city = 'seattle' THEN 'us-west1'
WHEN city = 'los angeles' THEN 'us-west1'
ELSE 'us-east1'
END
) STORED;
ALTER TABLE users ALTER COLUMN region SET NOT NULL;
ALTER TABLE users SET LOCALITY REGIONAL BY ROW AS "region";
```
I expect that the query below would get locality-optimized lookup join (since it's effectively doing the same thing as an example from [here](https://github.com/cockroachdb/cockroach/pull/93377#issue-1488270428)) but it doesn't:
```sql
EXPLAIN (OPT) SELECT * FROM user_promo_codes c INNER LOOKUP JOIN users u ON c.city = u.city AND c.user_id = u.id LIMIT 3;
```
```
inner-join (lookup users [as=u])
├── flags: force lookup join (into right side)
├── lookup columns are key
├── project
│ ├── locality-optimized-search
│ │ ├── scan user_promo_codes [as=c]
│ │ │ ├── constraint: /23/18/19/20: [/'us-east1' - /'us-east1']
│ │ │ └── limit: 3
│ │ └── scan user_promo_codes [as=c]
│ │ ├── constraint: /31/26/27/28
│ │ │ ├── [/'europe-west1' - /'europe-west1']
│ │ │ └── [/'us-west1' - /'us-west1']
│ │ └── limit: 3
│ └── projections
│ └── CASE WHEN c.city = 'amsterdam' THEN 'europe-west1' WHEN c.city = 'paris' THEN 'europe-west1' WHEN c.city = 'rome' THEN 'europe-west1' WHEN c.city = 'new york' THEN 'us-east1' WHEN c.city = 'boston' THEN 'us-east1' WHEN c.city = 'washington dc' THEN 'us-east1' WHEN c.city = 'san francisco' THEN 'us-west1' WHEN c.city = 'seattle' THEN 'us-west1' WHEN c.city = 'los angeles' THEN 'us-west1' ELSE 'us-east1' END
└── filters (true)
```
Perhaps it's the projection that is preventing that, or perhaps I'm misunderstanding something.
Note that initially I tried `CASE` statements for the new manually added columns without the `ELSE` part, in which case the `region` column could be NULL, but adding the `ELSE` didn't change anything (in theory, this shouldn't matter since we have NOT NULL constraint on the column).
cc @msirek
Jira issue: CRDB-26678
Contributor guide
Research direction
Reproduce the issue with the cockroach demo movr command and the supplied multi-region ALTER statements, then run the EXPLAIN (OPT) lookup-join query. Start by comparing the observed locality-optimized-search plan with the expected locality-optimized lookup join. Done means identifying and correcting why this plan is not selected, with the query producing the expected optimized plan.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go, sql
- Domain
- databases, distributed-systems
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100