apache / apache/datafusion

UNION schema depends on branch order in case of array values

Open
#12,292 4 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
Rust
Stars
9.3k
Forks
2.4k
Avg merge
3d 7h
Merged PRs (30d)
344

Description

### Describe the bug

```
> SELECT DISTINCT arrow_typeof(x[0]) FROM (SELECT make_array(2) x UNION ALL SELECT make_array(now()) x);
+---------------------------+
| arrow_typeof(x[Int64(0)]) |
+---------------------------+
| Int64 |
+---------------------------+
```

but same query with union branches swapped gives different return type

```
> SELECT DISTINCT arrow_typeof(x[0]) FROM (SELECT make_array(now()) x UNION ALL SELECT make_array(2) x);
+---------------------------------------+
| arrow_typeof(x[Int64(0)]) |
+---------------------------------------+
| Timestamp(Nanosecond, Some("+00:00")) |
+---------------------------------------+
```

### To Reproduce

```
SELECT DISTINCT arrow_typeof(x[0]) FROM (SELECT make_array(2) x UNION ALL SELECT make_array(now()) x);
SELECT DISTINCT arrow_typeof(x[0]) FROM (SELECT make_array(now()) x UNION ALL SELECT make_array(2) x);
```

### Expected behavior

The schema (output type) of `UNION ALL` should not depend on syntactic order of union's branches.

### Additional context

_No response_

Contributor guide

Open the contributing guide

Research direction

Run both UNION ALL queries from the issue and compare their output schemas and arrow_typeof results. Trace the UNION schema/type-resolution behavior for array values, then add coverage showing that swapping branch order produces the same output type and run the relevant DataFusion tests.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust, sql
Domain
data-engineering, databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.