DOC: Array interface protocol does not recognize bit field typestr and could mention AttributeError use
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 32.8k
- Forks
- 12.8k
- Avg merge
- 1d 7h
- Merged PRs (30d)
- 197
Description
Describe the issue:
I am trying to implement the array interface protocol for Polars Series. Polars represents its boolean arrays (and validity buffers) as bit-packed booleans, e.g. 8 booleans per byte. I figured I would need to specify a bit field (e.g. t type) to do this, however, it seems numpy does not recognize any way of defining a bit field. I've tried various combinations (with descr too) to get this to work, but I can't figure it out.
Could it be that NumPy does not handle bit field types correctly?
Reproduce the code example:
import numpy as np
class wrapper:
pass
interface = {
"shape": (3,),
"typestr": "|t1",
"version": 3,
}
wrapper.__array_interface__ = interface
result = np.array(wrapper, copy=False)
Error message:
Traceback (most recent call last):
File "/home/stijn/code/polars/py-polars/repro.py", line 19, in <module>
result = np.array(wrapper, copy=False)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: data type '|t1' not understood
Python and NumPy Versions:
1.26.3
3.12.0 (main, Nov 16 2023, 08:47:32) [GCC 11.4.0]
Runtime Environment:
[{'numpy_version': '1.26.3',
'python': '3.12.0 (main, Nov 16 2023, 08:47:32) [GCC 11.4.0]',
'uname': uname_result(system='Linux', node='frosty', release='6.1.0-1029-oem', version='#29-Ubuntu SMP PREEMPT_DYNAMIC Tue Jan 9 21:07:34 UTC 2024', machine='x86_64')},
{'simd_extensions': {'baseline': ['SSE', 'SSE2', 'SSE3'],
'found': ['SSSE3',
'SSE41',
'POPCNT',
'SSE42',
'AVX',
'F16C',
'FMA3',
'AVX2'],
'not_found': ['AVX512F',
'AVX512CD',
'AVX512_KNL',
'AVX512_KNM',
'AVX512_SKX',
'AVX512_CLX',
'AVX512_CNL',
'AVX512_ICL']}},
{'architecture': 'Prescott',
'filepath': '/home/stijn/code/polars/.venv/lib/python3.12/site-packages/numpy.libs/libopenblas64_p-r0-0cf96a72.3.23.dev.so',
'internal_api': 'openblas',
'num_threads': 12,
'prefix': 'libopenblas',
'threading_layer': 'pthreads',
'user_api': 'blas',
'version': '0.3.23.dev'}]
None
Context for the issue:
I am trying to improve interoperability between the Polars dataframe library and NumPy.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by running the provided array_interface reproduction and tracing NumPy's parsing of the "typestr" field. Determine whether bit-packed boolean types are supported and how the reported AttributeError usage should be addressed; done requires an agreed behavior with regression coverage or documentation.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- numpy, python
- Domain
- api
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100