Loading stack of images slow
- Dominant language
- Python
- Stars
- 226
- Forks
- 56
- PR merge metrics
- No merged PRs in 30d
Description
I have about 20k 500px500p images stored in a directory.
My ultimate goal is to reshape them into only a few images, blocking the 500s into the few larger images.
I first wanted to see how fast I could load the images.
```
arr=dask_image.imread.imread('*.png')[:100].compute()
```
versus
```
from PIL import Image
arr=[Image.open(img) for img in glob.glob('*.png')[:100]]
```
The second task is performing much much much faster than the first task (first task hangs).
Can you help me understand why this is so and what I can do to speed this up?
Chunk size is (1,500,500,3) but I changed it to (25,500,500,3) and it didn't really make a difference. I would normally expect this to be a fast and easy Dask operation. I even started processing and threaded distributed clients with no luck.
Help is very much appreciated. Thanks!
Contributor guide
Research direction
No repository files or tests are named. Start by reproducing the dask_image.imread.imread('*.png')[:100].compute() and PIL comparison, then inspect the effects of the reported chunk sizes and threaded or distributed clients. Done means explaining the slowdown and identifying a validated way to make this image-loading workflow faster.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- computer-vision, performance
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100