Add tests for support for `DataType::Union`
- Dominant language
- Rust
- Stars
- 9.3k
- Forks
- 2.4k
- Avg merge
- 3d 7h
- Merged PRs (30d)
- 344
Description
### Is your feature request related to a problem or challenge?
As we saw on https://github.com/apache/datafusion/pull/10139 with @joroKr21 there is not any particularly good way to test the coverage of [`DataType::Union`](https://docs.rs/arrow/latest/arrow/datatypes/enum.DataType.html#variant.Union) today with our tests.
Note this is different than the SQL `UNION` / `UNION ALL` feature in [union.slt](https://github.com/apache/datafusion/blob/16e3831734358d2c628c7ff281cddd680dc4aa10/datafusion/sqllogictest/test_files/union.slt)
### Describe the solution you'd like
I would like some way to test `DataType::Union` , ideally via sqllogictest -- in `union_type.slt`
### Describe alternatives you've considered
I think the ideal alternative is to create a function that creates a `UnionArray` from different columns and use that function to create some simple queries / data in `union_type.slt`
Something like `make_union(col1, col2, ...)` (similarly to [`make_array`](https://arrow.apache.org/datafusion/user-guide/sql/scalar_functions.html#make-array)) that results in a single column output of `UnionArray`
```sql
--- create a single UNION with two members (String and F64) with the string variant
select make_union('a'::varchar, null::double)
--- create a single UNION with two members (String and F64) with the f64
select make_union(null::varchar, 1.234)
--- Error: can't figure out which variant to create
select make_union('a', 123.4)
```
An alternate, option would be to register tables specially for similarly to https://github.com/apache/datafusion/blob/16e3831734358d2c628c7ff281cddd680dc4aa10/datafusion/sqllogictest/src/test_context.rs#L75-L78)
We could create a UnionArray using the examples from https://docs.rs/arrow/latest/arrow/array/struct.UnionArray.html
### Additional context
See https://github.com/apache/datafusion/pull/10139#pullrequestreview-2013390566
Contributor guide
Research direction
Start with the proposed union_type.slt and compare it with the existing union.slt. Read the sqllogictest test_context.rs registration area and the Arrow UnionArray examples to evaluate the suggested approaches. Done means DataType::Union has executable sqllogictest coverage, including the supported variants and invalid or ambiguous cases.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust, sql
- Domain
- backend, testing-qa
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100