Plan cannot be built when using virtual column and UNION ALL in SQL query
- Dominant language
- Java
- Stars
- 14.1k
- Forks
- 3.8k
- Avg merge
- 2d 58m
- Merged PRs (30d)
- 233
Description
# Affected Version
Version: 0.20.0-iap3 (imply build)
### Description
I ran into this problem when playing with sub-query UNION ALL requests, introduced in druid 0.20.0. It was working fine until I had to add a virtual column (`0 as expense`) to have the same columns in the two SELECT.
The easiest way I found to re-create this problem is the following.
```sql
SELECT COUNT(*)
FROM (
(SELECT 1 as a FROM "test-bug")
UNION ALL
(SELECT 2 as a FROM "test-bug")
)
```
```
Error: Unknown exception
Cannot build plan for query: SELECT count(*) FROM ( (SELECT 1 as a FROM "test-bug") UNION ALL (SELECT 2 as a FROM "test-bug") )
org.apache.druid.java.util.common.ISE
```
Stacktrace:

Note 1: the query works by replacing `SELECT count(*)` by `SELECT a`. And it doesn't work with any expression involving `a` (for eg: `SUM(a)`.
Note 2: The bug is not related to make the union over the same DS. It's just to simply the example. The original bug was found when doing an union between 2 different tables
"test-bug" is a basic data-source created from a kafka-ingestion task.

Note 3: Surprisingly, the same structure works when the union is performed on a system table.
```sql
SELECT COUNT(*)
FROM (
(SELECT 0 as a FROM INFORMATION_SCHEMA.COLUMNS)
UNION ALL
(SELECT -1000 as a FROM INFORMATION_SCHEMA.TABLES)
)
```
Note 4: a real use case that hits this problem is the following:
```sql
SELECT type, SUM(weight) as weight
FROM (
(SELECT type, weight FROM table1)
UNION ALL
(SELECT type, 1 as weight FROM table2) -- weight doesn't exist in table2
)
GROUP BY type
```
Did I touch something not currently supported by druid UNION ALL, or is it a bug in the query transformation?
Thanks
Contributor guide
Research direction
Reproduce the failure using the provided SQL UNION ALL queries with virtual columns and aggregate expressions, comparing them with the working system-table example. Trace the query transformation and planning path associated with these queries; done means the affected COUNT and SUM queries build and execute plans successfully.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java, sql
- Domain
- databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100