groupBy query: limit push down to segment scan is poor performance
- Dominant language
- Java
- Stars
- 14.1k
- Forks
- 3.8k
- Avg merge
- 2d 58m
- Merged PRs (30d)
- 233
Description
### Affected Version
0.17.1
### Description
When druid-0.17.1 is used, the performance of group by query is worse than druid-0.16.1. When
applylimitpushdowntosegment is set to false in query context, the performance returns to normal.
From this figure, we can see that the CPU consumption is in LimitedBufferHashGrouper.

query performance is according to query/time from historical log:
`{"query/time":13372,"query/bytes":6118553,"success":true,"identity":"allowAll"}`
@himanshug
Contributor guide
Research direction
Start by comparing groupBy execution with applylimitpushdowntosegment enabled and disabled in Druid 0.17.1, using the reported query/time and CPU evidence as a baseline. Investigate LimitedBufferHashGrouper and the segment-scan limit pushdown path; done means identifying and correcting the regression, with performance returning near the 0.16.1 or disabled-setting baseline.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- databases, performance
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100