citusdata / citusdata/citus

Improve colocation checks for reference tables / intermediate results

Open
#6,653 2 comments 0 reactions 0 assignees View on GitHub
bug sql
Dominant language
C
Stars
12.8k
Forks
794
Avg merge
2d 14h
Merged PRs (30d)
31

Description

```sql
SELECT t1.id
FROM (
SELECT t2.id
FROM (
SELECT t0.id
FROM tbl_dist1 t0
LIMIT 5
) AS t2
INNER JOIN tbl_dist1 AS t3 USING (id)
) AS t1
FULL JOIN tbl_dist1 t4 USING (id);
```

Due to LIMIT clause used in the innermost query, the query becomes the following after replacing that part of the query with intermediate result:
```sql
SELECT t1.id
FROM (
(
SELECT t2.id
FROM (
(
SELECT intermediate_result.id
FROM read_intermediate_result('143_1'::text, 'binary'::citus_copy_format) intermediate_result(id integer)
) t2
JOIN public.tbl_dist1 t3 USING (id)
)
)
t1 FULL JOIN public.tbl_dist1 t4 USING (id)
)
```

And during the [recursive call made to CreateDistributedPlan()](https://github.com/citusdata/citus/blob/44c387b978a51b0c0e87c7f9aec154cfc3041da1/src/backend/distributed/planner/distributed_planner.c#L1091-L1093), we decide that the intermediate result t2 and the distributed table t3 are not co-located. And as a result, we decide creating a subplan for t1 and it becomes a recurring rel, and hence we decide recursively planning the distributed table t4 because now the outer-most join becomes of the form `recurring FULL JOIN non-recurring`.

This is not just a performance issue but causes messing up the subplans and hence might cause returning incorrect result, as in #6636. This happens because the recursive planner is apparently not supposed to do anything during [its second pass](https://github.com/citusdata/citus/blob/44c387b978a51b0c0e87c7f9aec154cfc3041da1/src/backend/distributed/planner/distributed_planner.c#L1091-L1093).

For now, #6650 will introduce some ereport() calls for such cases; but it'd be better to improve the colocation checks for a better sql coverage.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.