COUNT DISTINCT on multiple columns produces wrong result.
- Dominant language
- Java
- Stars
- 6.1k
- Forks
- 1.5k
- Avg merge
- 1d 21h
- Merged PRs (30d)
- 189
Description
The query `SELECT count(DISTINCT name, score) FROM scores` produces wrong results. This happens because all arguments except the first one are ignored while constructing DISTINCTCOUNT Function in `AggregationFunctionFactory.java`.
One way to fix this is to using CONCAT function to concat all arguments into a single argument before creating the DISTINCTCOUNT function. Does this sound good? If so I will go ahead and put this in a PR. Any other suggestions?
```
diff --git a/pinot-core/src/main/java/org/apache/pinot/core/query/aggregation/function/AggregationFunctionFactory.java b/pinot-core/src/main/java/org/apache/pinot/core/query/aggregation/function/AggregationFunctionFactory.java
index 328560739d..b9e78fe098 100644
--- a/pinot-core/src/main/java/org/apache/pinot/core/query/aggregation/function/AggregationFunctionFactory.java
+++ b/pinot-core/src/main/java/org/apache/pinot/core/query/aggregation/function/AggregationFunctionFactory.java
@@ -159,7 +159,13 @@ public class AggregationFunctionFactory {
case MINMAXRANGE:
return new MinMaxRangeAggregationFunction(firstArgument);
case DISTINCTCOUNT:
- return new DistinctCountAggregationFunction(firstArgument);
+ if (arguments.size() == 1) {
+ return new DistinctCountAggregationFunction(firstArgument);
+ }
+
+ arguments.add(ExpressionContext.forLiteral(""));
+ return new DistinctCountAggregationFunction(ExpressionContext
+ .forFunction(new FunctionContext(FunctionContext.Type.TRANSFORM, "CONCAT", arguments)));
case DISTINCTCOUNTBITMAP:
return new DistinctCountBitmapAggregationFunction(firstArgument);
case SEGMENTPARTITIONEDDISTINCTCOUNT:
```
Contributor guide
Research direction
Start in pinot-core/src/main/java/org/apache/pinot/core/query/aggregation/function/AggregationFunctionFactory.java and reproduce SELECT count(DISTINCT name, score) FROM scores. Trace how DISTINCTCOUNT is built when multiple arguments are supplied and verify that the result accounts for every argument. Done means the query returns the correct distinct count without ignoring later arguments.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- databases
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 38/100