Performance issue with `sum`
- Dominant language
- Python
- Stars
- 226
- Forks
- 56
- PR merge metrics
- No merged PRs in 30d
Description
Working on a modest-size cube, I found that `scipy.ndimage.sum` is ~100-300x faster than `dask_image.ndmeasure.sum_labels`
```python
import numpy as np, scipy.ndimage
blah = np.random.randn(19,512,512)
msk = blah > 3
lab, ct = scipy.ndimage.label(msk)
%timeit scipy.ndimage.sum(msk, labels=lab, index=range(1, ct+1))
# 117 ms ± 2.85 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)
```
vs
```python
rslt = ndmeasure.sum_labels(msk, label_image=lab, index=range(1, ct+1))
rslt
# dask.array
rslt.compute()
# [########################################] | 100% Completed | 22.9s
```
Note also that the task creation takes nontrivial time:
```python
%timeit ndmeasure.sum_labels(msk, label_image=lab, index=range(1, ct+1))
# 15.4 s ± 2.02 s per loop (mean ± std. dev. of 7 runs, 1 loop each)
```
While I understand that there ought to be some cost to running this processing through a graph with dask, this seems excessively slow. Is there a different approach I should be taking, or is this a bug?
Contributor guide
Research direction
Start by reproducing the reported benchmarks for scipy.ndimage.sum and ndmeasure.sum_labels using the 19×512×512 example and the displayed label range. Inspect the sum_labels entry point and its Dask task creation, including the cost before compute. Done means establishing whether the overhead is expected or a performance bug and recording the relevant evidence or next step.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- performance
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100