apache / apache/arrow-rs

[Variant] Add benchmarks for typed unshredded variant_get

Open
#10,988 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Rust
Stars
3.6k
Forks
1.3k
Avg merge
2d 18h
Merged PRs (30d)
169

Description

### Is your feature request related to a problem or challenge?

`parquet-variant-compute/benches/variant_kernels.rs` currently benchmarks:

- Untyped extraction from an unshredded primitive Variant array
- Typed extraction from a perfectly shredded UTF-8 array
- Typed `Int32` extraction through a path in an unshredded object

It does not benchmark typed conversion of unshredded Variant values. Therefore, the benchmark suite cannot evaluate performance changes to these conversion paths or detect regressions introduced while aligning `variant_get` behavior with the Arrow cast kernels.

### Describe the solution you'd like

Add representative benchmarks for typed, unshredded `variant_get`.

The benchmarks should be organized around materially different execution paths and input shapes rather than creating a cross-product of Variant types and Arrow destination types.

Initial coverage should include the UTF-8 and Binary destination paths, with representative inputs that distinguish:

- Values that can be extracted directly
- Values that require conversion or formatting
- Nested values such as lists and objects

Inputs should be deterministic and large enough to produce stable measurements. Benchmark throughput should be reported alongside iteration time.

Add the benchmarks in a benchmark-only PR so they can be merged and used to compare the existing and proposed implementations.

### Describe alternatives you've considered

### Additional context

The existing benchmark suite contains useful controls for untyped extraction, perfect shredding, and path traversal. The new cases should complement those controls rather than duplicate them.

This came up while reviewing #10114.

Contributor guide

Open the contributing guide

Research direction

Read parquet-variant-compute/benches/variant_kernels.rs and its existing controls for untyped extraction, perfect shredding, and path traversal. Add deterministic, large inputs covering typed unshredded UTF-8 and Binary extraction, including direct values, conversion or formatting, and nested lists or objects. Done means the benchmark reports throughput and iteration time without duplicating existing cases.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
data, performance
Issue type
Feature
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
72/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.