apache / apache/druid

Regression in filtered aggregate for double columns

Open
#17,455 2 comments 0 reactions 0 assignees View on GitHub
Area - Querying Performance
Dominant language
Java
Stars
14.1k
Forks
3.8k
Avg merge
2d 58m
Merged PRs (30d)
233

Description

### Affected Version

Druid 29.0.1

### Description

```
SELECT
(TIME_FLOOR("__time", 'P1D', NULL, 'Asia/Calcutta')) AS "__time",
SUM(CASE WHEN "countryIsoCode" IN ('US') THEN added ELSE 0 END) AS "CASE_LONG",
SUM(CASE WHEN "countryIsoCode" IN ('US') THEN CAST(added as double) ELSE 0.0 END) AS "CASE_DOUBLE"
FROM "wikipedia"
WHERE ("__time" >= '2024-10-23T09:00:00.000Z' AND "__time" < '2024-10-30T11:00:00.000Z')
GROUP BY 1
ORDER BY "__time"
LIMIT 10001
```

If we check the query plan for above query, `CASE_LONG` aggregate gets converted to a filtered aggregator where as `CASE_DOUBLE` gets translated to a virtual column with `case_searched` expression which impacts the performance. Performance wise there is a big difference for real use case if there are multiple dimension values to check. Filtered aggregator is much faster than virtual column based aggregate with `case_searched` expression.

Contributor guide

Open the contributing guide

Research direction

Start by running the supplied SQL query on Druid 29.0.1 and comparing its query plan, especially the CASE_LONG and CASE_DOUBLE aggregates. Trace the planner path that translates filtered SUM expressions and determine why the double expression becomes a case_searched virtual column; done means equivalent double aggregates use the faster filtered-aggregator path without changing results.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, sql
Domain
data, databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.