apache / apache/arrow-rs

Separate pyo3 arrow integration into standalone crate

Open
#6,776 1 comment 1 reaction 0 assignees View on GitHub
enhancement
Dominant language
Rust
Stars
3.6k
Forks
1.3k
Avg merge
2d 18h
Merged PRs (30d)
169

Description

The `pyo3` crate released version `0.23`. The `arrow` crate depends on `pyo3 0.22` and it is not possible for arrow to update the `pyo3` dependency without a major breaking release. Therefore, it is currently not possible for a user to upgrade a codebase that depends both on `arrow` and `pyo3`.

To fix this, we could instead move the arrow pyo3 integration into a separate crate. This crate could then be updated to a new version of pyo3 outside of the arrow major release cadence. Since it is a separate crate, it would require wrapper structs around structs that implement `IntoPyObject`.

Contributor guide

Open the contributing guide

Research direction

Start by locating the arrow crate's pyo3 integration and its pyo3 dependency declaration. Review how the integration's IntoPyObject implementations are exposed and how crate boundaries affect users that depend on both arrow and pyo3. Done means the integration is independently versionable without requiring an arrow major release.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
build-system
Issue type
Refactor
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.