apache / apache/datafusion

panic: GROUP BY or SELECT DISTINCT over a zero-field (empty) struct key

Open
#25,281 0 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
Rust
Stars
9.3k
Forks
2.4k
Avg merge
3d 7h
Merged PRs (30d)
344

Description

### Describe the bug

`GROUP BY` (and `SELECT DISTINCT`) over a group key whose type contains a zero-field struct (`DataType::Struct(Fields::empty())`) panics when the aggregate emits.

A zero-field struct array has no child to derive its length from, so arrow-rs refuses `StructArray::try_new` for it. The group-value emit path rebuilds the key column with that constructor and unwraps the resulting error.

### To Reproduce

```console
$ datafusion-cli -c "select arrow_cast(null, 'Struct()') as s, count(*) from (values (1),(2),(3)) group by 1;"
DataFusion CLI v55.1.0

thread 'tokio-rt-worker' panicked at datafusion/physical-plan/src/aggregates/group_values/multi_group_by/row_backed.rs:225:14:
dictionary re-encode during emit: ArrowError(InvalidArgumentError("use StructArray::try_new_with_length or StructArray::new_empty_fields to create a struct array with no fields so that the length can be set correctly"), Some(""))
Error: Join Error
caused by
External error: task 63 panicked with message "dictionary re-encode during emit: ..."
```
Same panic for `SELECT DISTINCT`:

```sql
select distinct arrow_cast(null, 'Struct()') as s from (values (1),(2));
```

and when the zero-field struct is nested inside the key rather than being the whole key:

```sql
select named_struct('a', column1, 'z', arrow_cast(null, 'Struct()')) as s, count(*)
from (values (1),(2)) group by 1;
```

### Expected behavior

The query returns its groups. A zero-field struct is a valid Arrow type and carries one bit per row (null or not), so it should group like any other key.

### Additional context

Reproduced on `main` at 15f32dd7ac751952fffc53906144dfa67c5e868f.

Contributor guide

Open the contributing guide

Research direction

Start with datafusion/physical-plan/src/aggregates/group_values/multi_group_by/row_backed.rs at the reported emit path, then run the provided GROUP BY and SELECT DISTINCT reproductions. Done means zero-field structs, including nested keys, group successfully without a panic and return their expected rows.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
databases
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
76/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.