Poor numerical consistency with numpy for reductions with dtype=`float16` specified
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 668
- Forks
- 141
- Avg merge
- 2d 8h
- Merged PRs (30d)
- 4
Description
Reductions like COO.sum and COO.mean fail to match numpy in all cases when dtype=numpy.float16 is specified. For example:
import sparse
import numpy as np
x = np.array([[[ 0, 0, 4526, 0],
[ 0, 0, -37, 0],
[ 8372, 0, 7915, 0]],
[[ 0, 0, 0, 0],
[ 0, 0, -7917, 0],
[-9719, 0, 0, 0]]], dtype='i4')
s = sparse.COO.from_numpy(x)
res = s.sum(axis=(0, 2), dtype='f2')
sol = x.sum(axis=(0, 2), dtype='f2')
print(res.todense())
print(sol)
outputs:
[ 4530. -7950. 6564.]
[ 4530. -7950. 6570.]
It's not clear if each of these will need to be fixed per-method, or if there's a general fix in the reduce code. For sum, numpy interprets x.sum(dtype='f2') the same as x.astype('f2').sum(), but this is not true for x.mean(dtype='f2').
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start at the COO.sum and COO.mean reduction entry points and reproduce the provided float16 examples against NumPy. Trace how the dtype argument is handled for reductions over multiple axes, then verify that sparse results match the corresponding NumPy results for the shown cases.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- numpy, python
- Domain
- data
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100