Compound join conditions and IsJoinClause
- Dominant language
- C
- Stars
- 12.8k
- Forks
- 794
- Avg merge
- 2d 14h
- Merged PRs (30d)
- 31
Description
We recently ran into an issue related to compound join conditions (#58) in a private Slack channel. @robin900 looked deeper into the issue and noted that fixing the issue is tricky due to multiple code paths that do join planning. I'm copy/pasting his notes from the Slack chat below.
##
The two most common patterns for us, when there is a compound join condition, are:
1) `a JOIN b on a.dist_column = b.dist_column AND a.some_tstz <@ b.some_tstzrange`
2) `a JOIN b on a.dist_column = b.dist_column AND a.tstz < b.tstz`
We also have a rare `a JOIN b on a.dist_column = b.dist_column AND a.ctid < b.ctid` but we can work around the ctid problem.
My understanding, after reviewing the citus code, is that it would take significant refactoring to evaluate the list of clauses all at once, instead of `IsJoinClause()` called in series on a `clauseList`.
I note that the dummy join `a.fk_account_id = b.fk_account_id AND b.created_at = lower(a.during)` works merely because `IsJoinClause` returns true for both clauses.
They use `=` and left and right expr reference different tables (different `->varno`).
Reading the citus code, i see a confusion of what question the function `IsJoinClause` answers. This confusion leads to my confusion.
I see 2, perhaps 3, uses of the function:
1. In multi_join_order.c, to determine fixed join order for implicit joins; plus inside `ApplicableJoinClauses` where it’s just checking whether the left table and right table in each clause are applicable, and it’s not clear that equality operator is necessary.
2. In multi_logical_planner.c, to determine which clauses should participate in joins but not clear if equality operator is necessary for the logic to work.
3. A rough equivalent of `IsJoinClause` for lateral join optimization: https://github.com/citusdata/citus/blob/aa15043b0905a1fdba7f649d4986204c5efdf1e2/src/backend/distributed/planner/multi_logical_optimizer.c#L3431
Contributor guide
Research direction
Start by tracing IsJoinClause and ApplicableJoinClauses in multi_join_order.c and multi_logical_planner.c, then compare the related lateral join logic in multi_logical_optimizer.c around the referenced line. Determine which callers require equality operators and how compound clause lists are handled. Done means the join-planning paths consistently support the reported compound conditions, with tests or other verification described by the implementation.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- c, postgresql, sql
- Domain
- databases, distributed-systems
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100