apache / apache/datafusion

Nested field access on nullable structs discard parent nulls

Closed
#25,120 2 comments 0 reactions 1 assignee Claimed by @AdamGS View on GitHub
bug
Dominant language
Rust
Stars
9.3k
Forks
2.4k
Avg merge
3d 7h
Merged PRs (30d)
344

Description

### Describe the bug

If we have a nested struct array, we only take the leaf nullability into account, instead of of an `AND` of all parent nullability.

### To Reproduce

```rust
let inner_array = StructArray::new(
vec![Field::new("value", DataType::Int32, false)].into(),
vec![Arc::new(Int32Array::from(vec![1, 2, 3]))],
None,
);
// Only the outer struct marks row 1 as null; its children are all valid.
let outer_array = StructArray::new(
vec![Field::new("inner", inner_array.data_type().clone(), false)].into(),
vec![Arc::new(inner_array)],
Some(NullBuffer::from(vec![true, false, true])),
);
```

If we try and get `outer['inner']['value']`, we currently get `[1, 2, 3]` instead of `[1, null, 3]`.

### Expected behavior

_No response_

### Additional context

_No response_

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.