Proposal: support union all only by manipulating plan.
Nobody has claimed this yet.
- Dominant language
- C++
- Stars
- 1k
- Forks
- 423
- Avg merge
- 1d 15h
- Merged PRs (30d)
- 24
Description
To support "union all", there are two alternatives: support union all operator on TiFlash, or transfer the plans on TiDB.
For fast implemetation, I implement the union all by multiplying the plans. For expamle, given a sql:
(select * from x1) union all (select * from x2) union all (select * from x3)
It's equal to three (maybe more) flattened MPP tasks:
- select * from x1
- select * from x2
- select * from x3
We take the advantage that the final "MppGather" has taken the responsibilities of doing "union all". For a more complicated example:
select * from (select * from x1 union all select * from x2) X join (select * from x3 union all select * from x4) Y on X.a = Y.a
For this case, what first comes to mind is to do union all crossly, say union all (x1 join x3) (x1 join x4) (x2 join x3) (x2 join x4). That's a disaster if the branches of union increase. There is a way around it. If the subplan of join is union all, always do exchange before union all. For this statement, if we adopt shuffled hash join, the final tasks are:
1. (Exchange Recv(task 2, 3)) join (Exchange Recv(task 4, 5))
2. select * from x1 then Exchange Sender shuffling and sending data to task1
3. select * from x2 then Exchange Sender shuffling and sending data to task1
4. select * from x3 then Exchange Sender shuffling and sending data to task1
5. select * from x4 then Exchange Sender shuffling and sending data to task1
If we choose broadcast join, the tasks will be
1. x1 join (Exchange Recv(task 3, 4))
2. x2 join (Exchange Recv(task 3, 4))
3. select * from x3 then Exchange Sender broadcast data to task1, 2
5. select * from x4 then Exchange Sender broadcast data to task1, 2
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
The issue names no files or tests. Start by tracing TiFlash's MppGather and exchange handling for UNION ALL cases; done would require an agreed implementation and validation for the simple, shuffled-hash-join, and broadcast-join examples.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp, sql
- Domain
- databases, distributed-systems
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 30/100