Each Union branch should produce fields with `nullable: true` if any of the branches produces fields with `nullable: true` on the same position
- Dominant language
- Rust
- Stars
- 9.3k
- Forks
- 2.4k
- Avg merge
- 3d 7h
- Merged PRs (30d)
- 344
Description
### Describe the bug
It was found in #15242 that the nullability of `Literal` is simply based on its scalar value and does not take its nullability in the input schema into account.
### To Reproduce
This issue cannot be seen in the CLI, as it doesn't do any strict type checking of record batches when displaying results. However, it can be observed in SLTs where `normalize::convert_batches` ensures that each record batch's schema is the same as that of the first batch.
Take the following query for example:
```
select x, y, z from t3 union all by name select z, y, x, 'd' as zz from t3;
```
`UNION` is a case where record batch streams from each input will be merged into an output stream.
`zz` should be considered as nullable in the output schema because it is not found in the lhs. However, the current behavior is that record batches streamed from the rhs do not have `zz` as null because of the current implementation of `PhysicalExpr::nullable` for `Literal`.
### Expected behavior
The nullability of `Literal` should be determined by its surrounding context _and_ value, not only its value.
### Additional context
_No response_
Contributor guide
Research direction
Start with the PhysicalExpr::nullable implementation for Literal and the UNION schema-merging path described in the issue. Reproduce the query in an SLT and inspect normalize::convert_batches, then verify that the rhs batch's added zz field is nullable and schema normalization succeeds.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- databases
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100