apache / apache/arrow

[Python] Allow disabling more components

Open
#33,126 7 comments 0 reactions 0 assignees View on GitHub
Component: Python Type: enhancement
Dominant language
C++
Stars
17.1k
Forks
4.3k
Avg merge
3d 13h
Merged PRs (30d)
88

Description

Some users would like to build lightweight versions of PyArrow, for example for use in AWS Lambda or similar systems which constrain the total size of usable libraries.

However, PyArrow currently mandates some Arrow C++ components which can lead to a very sizable Arrow binary install: Compute, CSV, Dataset, Filesystem, HDFS and JSON.

**Reporter**: [Antoine Pitrou](https://issues.apache.org/jira/browse/ARROW-17916) / @pitrou
#### Related issues:
- [[C++] Split non-cast compute kernels into a separate shared library](https://github.com/apache/arrow/issues/25025) (is related to)

**Note**: *This issue was originally created as [ARROW-17916](https://issues.apache.org/jira/browse/ARROW-17916). Please see the [migration documentation](https://github.com/apache/arrow/issues/14542) for further details.*

Contributor guide

Open the contributing guide

Research direction

Start by reviewing how PyArrow currently mandates the listed Arrow C++ components: Compute, CSV, Dataset, Filesystem, HDFS and JSON. The related issue on splitting compute kernels may provide context; done means users can build lightweight PyArrow versions without components they do not need.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp, python
Domain
build-system
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.