ClickHouse / ClickHouse/ClickHouse

DISTINCT tuple(a, b) doesn't use DISTINCT in order optimization

Open
#39,367 0 comments 0 reactions 0 assignees View on GitHub
comp-query-optimizer performance
Dominant language
C++
Stars
49.9k
Forks
9k
Avg merge
21h 32m
Merged PRs (30d)
515

Description

**Describe the situation**
DISTINCT tuple() doesn't use DISTINCT in order optimization for preliminary distinct

**How to reproduce**

EXPLAIN PIPELINE

SELECT DISTINCT (CounterID, EventDate)
FROM hits_v1
ORDER BY
CounterID ASC,
EventDate ASC

Query id: 444a1a18-699a-4849-aa8c-de0d5814e8ab

┌─explain───────────────────────────────┐
│ (Expression) │
│ ExpressionTransform │
│ (Distinct) │
│ DistinctTransform │
│ (Sorting) │
│ MergingSortedTransform 19 → 1 │
│ (Distinct) │
DistinctTransform × 19 │
│ (Expression) │
│ ExpressionTransform × 19 │
│ (ReadFromMergeTree) │
│ MergeTreeInOrder × 19 0 → 1 │
└───────────────────────────────────────┘

EXPLAIN PIPELINE

SELECT DISTINCT CounterID, EventDate
FROM hits_v1

Query id: ed594f00-f7df-40fe-b14a-afc5531f943c

┌─explain─────────────────────────────────┐
│ (Expression) │
│ ExpressionTransform │
│ (Distinct) │
│ DistinctTransform │
│ Resize 16 → 1 │
│ (Distinct) │
DistinctSortedChunkTransform × 16 │
│ (Expression) │
│ ExpressionTransform × 16 │
│ (ReadFromMergeTree) │
│ MergeTreeThread × 16 0 → 1 │
└─────────────────────────────────────────┘

### Cause
Here, the vector with column names in DISTINCT contains one element `{"tuple(CounterID,EventDate)"}` instead of 2 elements `{"CounterID","EventDate"}` which doesn't match sort description of input stream. Sounds very similar to #38772 😕
https://github.com/ClickHouse/ClickHouse/blob/07210c7b550b2517f7229cd0f5cb7ecd6d5874f1/src/Processors/QueryPlan/DistinctStep.cpp#L94

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.