apache / apache/arrow-rs

[Variant] Support composite typed_value values in VariantArray::try_value

Open
#10,620 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Rust
Stars
3.6k
Forks
1.3k
Avg merge
2d 18h
Merged PRs (30d)
169

Description

### Is your feature request related to a problem or challenge?

`VariantArray::try_value` can materialize primitive `typed_value` values but not legal shredded Struct or List values.

#10597 prevents these values from being silently reported as `Variant::Null`, but callers still cannot retrieve the complete row as a `Variant` without unshredding the array first.

### Describe the solution you'd like

Support reconstructing perfectly and partially shredded Struct/List values in `try_value`.

This likely requires an owned or COW representation because composite values must be encoded into new buffers, while `Variant` currently borrows its encoded bytes.

### Describe alternatives you've considered

Call `unshred_variant` on the complete array before accessing individual values. This works but requires array-level reconstruction when only one row may be needed.

### Additional context

- #10597
- Current unsupported arm: https://github.com/apache/arrow-rs/blob/main/parquet-variant-compute/src/variant_array.rs#L1126

Contributor guide

Open the contributing guide

Research direction

Start in parquet-variant-compute/src/variant_array.rs at the unsupported arm around line 1126, and read issue #10597 for the related behavior. Trace how try_value handles primitive typed_value values and how composite Struct/List values would need owned or copy-on-write encoded storage. Done means perfectly and partially shredded composite values can be retrieved as complete Variants without first unshredding the whole array.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
data
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.