apache / apache/arrow-rs

[EPIC] [Parquet] Stabilize Variant support (remove the variant_experimental flag)

Open
#10,546 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Rust
Stars
3.6k
Forks
1.3k
Avg merge
2d 16h
Merged PRs (30d)
168

Description

**Is your feature request related to a problem or challenge? Please describe what you are trying to do.**

Successor to https://github.com/apache/arrow-rs/issues/6736 (initial implementation) and https://github.com/apache/arrow-rs/issues/8480 (finalization). Most of the work tracked there is done, so this epic tracks only what gates renaming `variant_experimental` to `variant` (https://github.com/apache/arrow-rs/issues/8297). Non-blocking follow-on work is tracked separately in https://github.com/apache/arrow-rs/issues/10547.

**Bugs and spec conformance**
- [ ] https://github.com/apache/arrow-rs/pull/10352 (fixes 6 fuzzing findings, the two below are its follow-ups)
- [ ] https://github.com/apache/arrow-rs/issues/10359
- [ ] https://github.com/apache/arrow-rs/issues/10360
- [ ] https://github.com/apache/arrow-rs/issues/11050
- [x] https://github.com/apache/arrow-rs/issues/10145
- [x] https://github.com/apache/arrow-rs/issues/10315 (PR open: https://github.com/apache/arrow-rs/pull/10320)

**Cast semantics**
- [ ] https://github.com/apache/arrow-rs/issues/8982
- [ ] https://github.com/apache/arrow-rs/issues/10024
- [ ] https://github.com/apache/arrow-rs/issues/10102

**Feature completeness and API finalization**
- [x] https://github.com/apache/arrow-rs/issues/8477 (union support still open)
- [ ] https://github.com/apache/arrow-rs/issues/8628

**The finale**
- [ ] https://github.com/apache/arrow-rs/issues/8297
- [ ] https://github.com/apache/arrow-rs/issues/8566

Contributor guide

Open the contributing guide

Research direction

This is an epic rather than a single code change. Start by reading the linked bug, cast-semantics, feature-completeness, and API-finalization issues, along with PRs 10352 and 10320. Done means the listed blocking work is resolved and issue 8297 can rename variant_experimental to variant; follow-on work is tracked in issue 10547.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
data-engineering
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.