Inconsistent elemwise auto-densify rules
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 668
- Forks
- 141
- Avg merge
- 2d 8h
- Merged PRs (30d)
- 4
Description
Describe the bug
Sometimes element wise operations return a dense result, sometimes they error. It seems to depend on the broadcasting.
To Reproduce
import sparse
import numpy as np
s = sparse.random((20, 10), .5)
dense_mtx = np.random.rand(20, 10)
dense_vec = np.random.rand(10)
print(type(s + dense_mtx))
# <class 'numpy.ndarray'>
s + dense_vec
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
<ipython-input-1-8f24377c4243> in <module>
9 # np.ndarray
10
---> 11 s + dense_vec
/usr/local/lib/python3.8/site-packages/numpy/lib/mixins.py in func(self, other)
19 if _disables_array_ufunc(other):
20 return NotImplemented
---> 21 return ufunc(self, other)
22 func.__name__ = '__{}__'.format(name)
23 return func
~/github/sparse/sparse/_sparse_array.py in __array_ufunc__(self, ufunc, method, *inputs, **kwargs)
303
304 if method == "__call__":
--> 305 result = elemwise(ufunc, *inputs, **kwargs)
306 elif method == "reduce":
307 result = SparseArray._reduce(ufunc, *inputs, **kwargs)
~/github/sparse/sparse/_umath.py in elemwise(func, *args, **kwargs)
47 """
48
---> 49 return _Elemwise(func, *args, **kwargs).get_result()
50
51
~/github/sparse/sparse/_umath.py in __init__(self, func, *args, **kwargs)
460
461 self._check_broadcast()
--> 462 self._get_fill_value()
463
464 def get_result(self):
~/github/sparse/sparse/_umath.py in _get_fill_value(self)
554 equivalent_fv = equivalent(fill_value, fill_value_array).all()
555 if not equivalent_fv and self.shape != self.ndarray_shape:
--> 556 raise ValueError(
557 "Performing a mixed sparse-dense operation that would result in a dense array. "
558 "Please make sure that func(sparse_fill_values, ndarrays) is a constant array."
ValueError: Performing a mixed sparse-dense operation that would result in a dense array. Please make sure that func(sparse_fill_values, ndarrays) is a constant array.
Expected behavior
Consistency. I'd like a dense output for this operation (following https://github.com/pydata/sparse/issues/10#issuecomment-356925562), but would be happy with the consistency of:
- Always error
- Always return dense
- Always return sparse
System
- OS and version: macOS 10.15
sparseversion (sparse.__version__) 0.12.0+8.g75125cd- NumPy version (
np.__version__) 1.20.2 - Numba version (
numba.__version__) 0.53.1
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Reproduce the two operations from the issue, then read sparse/_sparse_array.py around array_ufunc and sparse/_umath.py around elemwise, _check_broadcast, and _get_fill_value. Compare the broadcasting paths and existing behavior before deciding on the consistent result policy. Done means mixed sparse-dense elementwise operations follow one documented, tested outcome.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- numpy, python
- Domain
- data
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100