Unhandled Query Planning Failure when working with a VALUES query with a column full of NULLs when there is also other processing happening.
- Dominant language
- Java
- Stars
- 14.1k
- Forks
- 3.8k
- Avg merge
- 2d 58m
- Merged PRs (30d)
- 233
Description
The SQL data loader in the web console uses a VALUES query to make a sample dataset for easy previewing. I know the description might make this sounds like a crazy corner case but in actuality this is a very common thing to stumble upon as in a sample of data (20 rows or so) with many column you are very likely to get a column that is all NULL.
### Affected Version
All recent Druid versions that I tested including the latest build on master as of this writing and 30.0.0 RC.
### Description
Here is a very simple (self contained) query that fails:
```sql
SELECT
CAST("c1" AS VARCHAR) AS "channel",
CAST("c2" AS VARCHAR) AS "cityName",
PARSE_JSON("c3") AS "j"
FROM (
VALUES
('ca', NULL, '{}'),
('de', NULL, '{"a":"1"}'),
('de', null, '{"a":"2"}')
) AS "t" ("c1", "c2", "c3")
```
All that is logged in the broker is:
```
2024-05-15T18:43:43,102 WARN [sql[8f567aa7-38c4-4d27-89dc-60189e66f5c9]] org.apache.druid.sql.http.SqlResource - Exception while processing sqlQueryId[8f567aa7-38c4-4d27-89dc-60189e66f5c9] (org.apache.druid.error.DruidException: Unhandled Query Planning Failure, see broker logs for details)
```
Which is not helpful
Curiously if one of the NULLs is changed to, say a `''` it works:
Also if we remove the `PARSE_JSON` it also works:
From playing around with this it appears that there is some additional processing step that applying a function like `PARSE_JSON` adds that can not handle a NULL typed column that is being cast.
Contributor guide
Research direction
Start by running the self-contained VALUES query from the issue and inspecting the broker logs around the Unhandled Query Planning Failure, comparing it with the versions that remove PARSE_JSON or replace a NULL. The issue names no source files or tests; done means the query plans successfully with an all-NULL column and regression coverage protects this case.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java, sql
- Domain
- database
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 42/100