Regression in filtered aggregate for double columns
- Dominant language
- Java
- Stars
- 14.1k
- Forks
- 3.8k
- Avg merge
- 2d 58m
- Merged PRs (30d)
- 233
Description
### Affected Version
Druid 29.0.1
### Description
```
SELECT
(TIME_FLOOR("__time", 'P1D', NULL, 'Asia/Calcutta')) AS "__time",
SUM(CASE WHEN "countryIsoCode" IN ('US') THEN added ELSE 0 END) AS "CASE_LONG",
SUM(CASE WHEN "countryIsoCode" IN ('US') THEN CAST(added as double) ELSE 0.0 END) AS "CASE_DOUBLE"
FROM "wikipedia"
WHERE ("__time" >= '2024-10-23T09:00:00.000Z' AND "__time" < '2024-10-30T11:00:00.000Z')
GROUP BY 1
ORDER BY "__time"
LIMIT 10001
```
If we check the query plan for above query, `CASE_LONG` aggregate gets converted to a filtered aggregator where as `CASE_DOUBLE` gets translated to a virtual column with `case_searched` expression which impacts the performance. Performance wise there is a big difference for real use case if there are multiple dimension values to check. Filtered aggregator is much faster than virtual column based aggregate with `case_searched` expression.
Contributor guide
Research direction
Start by running the supplied SQL query on Druid 29.0.1 and comparing its query plan, especially the CASE_LONG and CASE_DOUBLE aggregates. Trace the planner path that translates filtered SUM expressions and determine why the double expression becomes a case_searched virtual column; done means equivalent double aggregates use the faster filtered-aggregator path without changing results.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java, sql
- Domain
- data, databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100