pingcap / pingcap/tidb

[expression] WITH ROLLUP can merge collation-distinct expressions and persist wrong summaries

Open
#70,113 2 comments 0 reactions 0 assignees View on GitHub
affects-7.5 affects-8.1 affects-8.5 component/expression found-by-ai severity/critical type/bug
Dominant language
Go
Stars
40.5k
Forks
6.2k
PR merge metrics
PR metrics pending

Description

## Bug Report

### 1. Minimal reproduce step (Required)

**Environment**

ROLLUP over same input with different CAST target metadata

**Steps**

Create a utf8mb4_bin source with a,A,b. INSERT the result of GROUP BY v COLLATE utf8mb4_general_ci, v COLLATE utf8mb4_bin WITH ROLLUP into a summary table. It succeeds with zero warnings and writes 5 rows: duplicated a/a and b/b detail-shaped totals plus grand total. Explicit UNION ALL of detail, subtotal, and grand-total definitions writes 6 rows, including distinct A/A and a/a details and two g2=1 subtotals. Reversing grouping expression order returns 7 rows; plain GROUP BY returns the correct 3 details.

### 2. What did you expect to see? (Required)

ROLLUP must preserve both collation-distinct grouping expressions and produce 3 detail, 2 subtotal, and 1 grand-total rows. Canonical equality must include every target-type field that changes value or equality semantics.

### 3. What did you see instead? (Required)

The bad plan projects only the general_ci CAST once, uses the same column twice in HashAgg/Expand, writes 5 rows with no subtotal level, and loses the binary A detail. Both bad and reference tables pass ADMIN CHECK TABLE.

### 4. What is your TiDB version? (Required)

ROLLUP over same input with different CAST target metadata

Likely root cause and fix direction

**Likely root cause**

COLLATE expressions normalize to CAST. simpleCanonicalizedHashCode appends only RetType.EvalType() for CAST, omitting collation, charset, flen, decimal, flags, and exact type. DeduplicateGbyExpression treats matching canonical bytes as semantic equality, and LogicalExpand substitutes both grouping items with one projected column.

**Fix direction**

Include full CAST target FieldType semantics in the canonical key, or use exact semantic equality after hash lookup. A counterfactual key containing type, flags, flen, decimal, charset, and collation preserves both projected columns and makes focused plus real-TiKV tests GREEN.

Contributor guide

Open the contributing guide

Research direction

Start with simpleCanonicalizedHashCode and DeduplicateGbyExpression, then trace how LogicalExpand and HashAgg/Expand use the canonical key. Add coverage for the collation-distinct ROLLUP reproduction and run the focused and real-TiKV tests. Done means both grouping expressions remain projected separately and the results contain 3 detail, 2 subtotal, and 1 grand-total row.

Written by the indexing model from the issue text.

Assessment

Tech stack
go, sql
Domain
databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
50/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.