Some dtypes not supported by numba
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 668
- Forks
- 141
- Avg merge
- 2d 8h
- Merged PRs (30d)
- 4
Description
Describe the bug
Big-endian floats (and maybe others) are not supported by numba, which leads to TypingError on some sparse operations.
To Reproduce
import numpy as np
import sparse as sp
little = np.array([1, 1, 1, 1, 1, 1], dtype='<f8')
big = np.array([1, 1, 1, 1, 1, 1], dtype='>f8')
Anp = np.random.randint(100, size=(4, 6)) / 100
A = sp.COO.from_numpy(Anp)
A.dot(little) # works
A.dot(big) # fails
The other way around is the same;
Abig = sp.COO.from_numpy(Anp.astype('>f8'))
Abig.dot(little) # fails
Abig.dot(big) # fails
Expected behavior
I expected the operation to work in all cases.
Could on-the-fly byte swap be implemented? May be with a warning?
In this specific case, on-the-fly byte swapping is what numpy does:
r = Anp.dot(big)
r.dtype # <f8, little-endian
For the case with Abig, where the sparse array itself is in a unsupported dtype, if not on-the-fly conversion is done, maybe there could be an issue raised, or at least a warning, on creation? To signify that the input array doesn't support the full features of sparse.
System
- OS and version: Ubuntu 18.04
sparseversion : 0.13.0- NumPy version : 1.21.2
- Numba version : 0.53.1
Additional context
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by running the provided Python reproduction with NumPy 1.21.2, Numba 0.53.1, and sparse 0.13.0 to compare little- and big-endian inputs in COO dot operations. Trace the resulting TypingError and determine the supported behavior for byte-swapping or rejecting unsupported dtypes; done means the reported cases behave consistently with the agreed warning or error behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- numpy, python
- Domain
- data
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100