apache / apache/datafusion

Error aliasing on double unnest on List[Struct]

Open
#12,162 1 comment 0 reactions 0 assignees View on GitHub
bug
Dominant language
Rust
Stars
9.3k
Forks
2.4k
Avg merge
3d 7h
Merged PRs (30d)
344

Description

### Describe the bug

Using a datafusion table with a column (`col`) of type `List[struct[]]`. Calling
```sql
SELECT unnest(unnest(col)) as col_name
FROM tbl
```

Results in the error

```shell
Internal error: unnest on struct can only be applied at the root level of select expression.
This was likely caused by a bug in DataFusion's code and we would welcome that you file an bug report in our issue tracker
```

### To Reproduce

Context
```
sql> describe test_tbl
+-------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-------------+
| column_name | data_type | is_nullable |
+-------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-------------+
| id | Utf8 | YES |
| labels | List(Field { name: "item", data_type: Struct([Field { name: "name", data_type: Utf8, nullable: true, dict_id: 0, dict_is_ordered: false, metadata: {} }]), nullable: true, dict_id: 0, dict_is_ordered: false, metadata: {} }) | YES |
+-------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-------------+
```
Without Alias
```
sql> select unnest(unnest(labels)) from test_tbl limit 3
+-----------------------------------------+
| unnest(unnest(test_tbl.labels)).name |
+-----------------------------------------+
| kind/bug |
| kind/enhancement |
| kind/enhancement |
+-----------------------------------------+
```

With Alias
```
sql> select unnest(unnest(labels)) as a from test_tbl limit 3
Status { code: Internal, message: "Internal error: unnest on struct can only be applied at the root level of select expression.\nThis was likely caused by a bug in DataFusion's code and we would welcome that you file an bug report in our issue tracker", metadata: MetadataMap { headers: {"content-type": "application/grpc", "date": "Mon, 26 Aug 2024 00:25:51 GMT", "content-length": "0"} }, source: None }
```

### Expected behavior

```
sql> select unnest(unnest(labels)) as an_alias from test_tbl limit 3
+-----------------------------------------+
| an_alias.name |
+-----------------------------------------+
| kind/bug |
| kind/enhancement |
| kind/enhancement |
+-----------------------------------------+
```

### Additional context

For outputs, using [spiceai](https://github.com/spiceai/spiceai/), specifically [spice sql](https://github.com/spiceai/spiceai/tree/trunk/bin/spice).

Contributor guide

Open the contributing guide

Research direction

Start by reproducing the nested UNNEST query with and without the alias, using the spice sql context described in the issue. Trace the alias handling for UNNEST over List[Struct]; the fix is complete when the aliased query returns the expected struct field output instead of the internal error.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust, sql
Domain
databases
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.