pingcap / pingcap/tidb

group_concat(... ORDER BY ...) returns elements that are not in the table when the result is truncated at group_concat_max_len and sort keys tie

Open
#71,187 4 comments 0 reactions 0 assignees View on GitHub
affects-25.10 affects-26.3 affects-7.5 affects-8.1 affects-8.5 contribution first-time-contributor severity/critical sig/execution type/bug
Dominant language
Go
Stars
40.5k
Forks
6.2k
PR merge metrics
PR metrics pending

Description

## Bug Report

### 1. Minimal reproduce step

```sql
CREATE TABLE gc (id INT, c VARCHAR(10));
INSERT INTO gc VALUES (3,'aaa'),(1,'a'),(0,'aaa'),(5,'ab'),(4,'x'),(2,'a'),(6,'x');
SET SESSION group_concat_max_len = 7;
SELECT group_concat(c ORDER BY c SEPARATOR ',') FROM gc;
```

The table holds two `'a'`, two `'aaa'`, one `'ab'` and two `'x'`. The sorted
concatenation is `a,a,aaa,aaa,ab,x,x`, so the first 7 bytes are `a,a,aaa`.

The same defect is reachable **without changing any setting**, at the default
`group_concat_max_len = 1024`, with a few hundred rows of tied short values:

```sql
CREATE TABLE g (id INT, c VARCHAR(10));
INSERT INTO g SELECT n, CASE WHEN n%6<3 THEN 'aaa' WHEN n%6=3 THEN 'a' ELSE 'b' END
FROM (WITH RECURSIVE s(n) AS (SELECT 1 UNION ALL SELECT n+1 FROM s WHERE n<600)
SELECT n FROM s) x;
-- the table holds exactly 100 x 'a', 300 x 'aaa', 200 x 'b'
SELECT group_concat(c ORDER BY c SEPARATOR ',') FROM g;
```

### 2. What did you expect to see?

The truncated result should be a prefix of the sorted concatenation, which is what
MySQL 9.7.2 returns for both queries.

Minimal case:

```
a,a,aaa
```

600-row case at the default 1024 bytes, MySQL 9.7.2 (counted by splitting the
result on the separator):

```
elements = 307 'a' = 100 'aaa' = 206 'b' = 0
```

i.e. all 100 `'a'` rows, then as many `'aaa'` as fit.

### 3. What did you see instead

Minimal case, TiDB v8.5.8 and master nightly:

```
a,a,a,a
```

Four `'a'` elements, although the table contains only two. The result is not a
prefix of the sorted concatenation, and `'a'` values appear that were never in the
table -- they are `'aaa'` rows whose text was truncated but which kept their
original sort key.

600-row case at the **default** `group_concat_max_len = 1024`, TiDB v8.5.8:

```
elements = 320 'a' = 127 'aaa' = 192 'b' = 0
```

27 `'a'` elements more than the table contains. Only warning 1260 ("Some rows were
cut by GROUPCONCAT") is raised; nothing indicates the result is wrong.

Under a hash join with concurrency > 1 the output additionally changes between
runs, because which tied row gets truncated depends on arrival order.

### 4. What is your TiDB version?

```
Release Version: v8.5.8
Edition: Community
Git Commit Hash: 8b857efa20363d50a8fa2ea7dd9809a85a61b115
Git Branch: HEAD
UTC Build Time: 2026-08-27 07:20:33
GoVersion: go1.25.12
Race Enabled: false
Check Table Before Drop: false
Store: unistore
```

Also reproduced on master nightly `v9.0.0-beta.2.pre-2261-g5000159a55`
(`pingcap/tidb:nightly`, pulled 2026-09-16): the minimal case returns `a,a,a,a`
there too.

---

### Suspected cause (from reading pkg/executor/aggfuncs/func_group_concat.go)

`topNRows.tryToAdd` keeps rows in a max-heap keyed by the ORDER BY items. When the
accumulated size exceeds `group_concat_max_len` it pops the largest row, and if that
row's buffer is longer than the excess it **truncates the buffer and pushes the row
back**. The truncated row keeps its original sort key, so with tied keys it is no
longer necessarily the heap maximum -- a different row with the same key can be
popped next. A row whose text was cut then survives in the interior of the result,
and rows that should have been dropped are kept.

Contributor guide

Open the contributing guide

Research direction

Start in pkg/executor/aggfuncs/func_group_concat.go, especially topNRows.tryToAdd and the heap handling described in the report. Run the minimal SQL reproduction and the 600-row case, then add or update regression coverage for tied sort keys and truncation. Done means results are valid prefixes of the sorted input, with no fabricated elements or run-to-run changes.

Written by the indexing model from the issue text.

Assessment

Tech stack
go, sql
Domain
databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
72/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.