apache / apache/arrow-rs

[Variant] Reconcile FixedSizeList support across shredded Variant APIs

Open
#10,616 2 comments 0 reactions 1 assignee Claimed by @cakeni View on GitHub
Dominant language
Rust
Stars
3.6k
Forks
1.3k
Avg merge
2d 18h
Merged PRs (30d)
169

Description

### Describe the bug

`shred_variant` and `unshred_variant` support `FixedSizeList`, but other shredded Variant APIs do not:

- `VariantArray::try_new` rejects `FixedSizeList` as an illegal `typed_value`.
- `variant_get` does not include `FixedSizeList` in indexed list traversal.

A `FixedSizeList` VariantArray can therefore be created by one public API but rejected or mishandled by another.

### To Reproduce

1. Shred a Variant list using a `DataType::FixedSizeList` schema.
2. Convert the result to its Arrow representation.
3. Reconstruct it using `VariantArray::try_new`, or access an element using an indexed `variant_get` path.

### Expected behavior

`FixedSizeList` should either be supported consistently across construction, validation, unshredding, and path traversal, or rejected consistently by the shredding APIs.

### Additional context

- Shredding: https://github.com/apache/arrow-rs/blob/main/parquet-variant-compute/src/shred_variant.rs#L161
- Unshredding: https://github.com/apache/arrow-rs/blob/main/parquet-variant-compute/src/unshred_variant.rs#L282
- Validation: https://github.com/apache/arrow-rs/blob/main/parquet-variant-compute/src/variant_array.rs#L1212
- Path traversal: https://github.com/apache/arrow-rs/blob/main/parquet-variant-compute/src/variant_get.rs#L160

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.