apache / apache/arrow-rs

No way to downcast Arc<dyn Array> to Arc<T: Array>

Open
#8,794 3 comments 3 reactions 0 assignees View on GitHub
enhancement
Dominant language
Rust
Stars
3.6k
Forks
1.3k
Avg merge
2d 14h
Merged PRs (30d)
167

Description

**Is your feature request related to a problem or challenge? Please describe what you are trying to do.**

arrow-rs makes heavy use of `ArrayRef` (= `Arc`), and it's impossible to impl Array for !Any types because [Array::as_any](https://docs.rs/arrow/latest/arrow/array/trait.Array.html#tymethod.as_any) is a required method. But for some reason we have `Array: Send + Sync` instead of `Array: Any + Send + Sync`, so there's no way to [Arc::downcast](https://doc.rust-lang.org/stable/std/sync/struct.Arc.html#method.downcast) an `ArrayRef` to e.g. `Arc`.

**Describe the solution you'd like**

```rust
trait Array: std::any::Any + ... { ... }
```

Which allows
```rust
let array: ArrayRef = ...;
let struct_array: Arc = Arc::downcast(array).map_err(...)?;
```

One big problem tho -- the blanket `impl Array for &T` means `Array` cannot require `Any` (because `Any: 'static`). I'm not sure how to work around that.

**Describe alternatives you've considered**

As far as I can tell, with today's code you have to downcast to `&StructArray`, clone it, and allocate a new Arc:
```rust
let array: ArrayRef = ...;
let struct_array = array.as_struct_opt().ok_or_else(...)?;
let struct_array = Arc::new(struct_array.clone());
```
Sure, cloning an array is _probably_ fairly inexpensive, but there's no way it's as cheap as an arc clone.

Contributor guide

Open the contributing guide

Research direction

Start by reading the Array trait, its as_any requirement, the ArrayRef alias, and the blanket impl for references described in the issue. Reproduce the Arc downcast limitation and compare it with the existing reference-downcast and clone workaround. Done means an agreed API design that permits the intended Arc downcast without breaking supported Array implementations, with tests covering the behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
data
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.