apache / apache/arrow

[Python] pyarrow.array should special-case array.array objects

Open
#38,137 2 comments 0 reactions 0 assignees View on GitHub
Component: Python Type: enhancement
Dominant language
C++
Stars
17.1k
Forks
4.3k
Avg merge
3d 13h
Merged PRs (30d)
88

Description

### Describe the enhancement requested

```python
import random
import array
import numpy as np
import pyarrow as pa
x = array.array("B", [random.randint(0, 255) for i in range(50000)])
```

`%timeit pa.array(x, pa.uint8())`: 2.5 ms ± 64 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)
`%timeit pa.array(np.asarray(x), pa.uint8())`: 6.79 µs ± 45.4 ns per loop (mean ± std. dev. of 7 runs, 100,000 loops each)

Converting to a numpy array gives a 350x speedup. `array.array` isn't commonly used but it is part of the Python standard library, so should probably be supported by `pa.array`.

### Component(s)

Python

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.