dask / dask/dask-image

For some schedulers, setting PIMS image reader's `.class_priority` is ineffective in controlling `dask-image.imread()`

Open
#262 15 comments 1 reaction 0 assignees View on GitHub
Dominant language
Python
Stars
226
Forks
56
PR merge metrics
No merged PRs in 30d

Description

cc: @jmdelahanty

Hi dask-image developers!

Normally an end-user may control which reader `pims.open()` uses to load images by simply increasing the `.class_priority` attribute of their preferred `pims` reader prior to calling `pims.open()`. [See this link.](http://soft-matter.github.io/pims/dev/custom_readers.html#plugging-into-pims-s-open-function)

```python
pims.ImageIOReader.class_priority = 100 # we set this very high in order to force pims.open() to use this reader
rgb_frames = pims.open('/path/to/video/file.mpg') # uses ImageIOReader
```

Since `dask-image.imread()` uses `pims.open()`, it would be great if it could mirror such functionality too.
```python
pims.ImageIOReader.class_priority = 100 # we set this very high in order to force dask's imread() to use this reader [via pims.open()]
rgb_frames = dask_image.imread.imread('/path/to/video/file.mpg') # uses ImageIOReader
```

And indeed this functionality *does* work for `dask-image.imread()` in single-machine schedulers, like "threading" and "sync". But I do not know of a way to make all processes, in a multi-process scheduler, for example, aware of the preferred reader's increased `.class_priority`. Any help here would be greatly appreciated.

Alternatively, it might be an idea to modify `dask-image.imread()` to receive a "reader" keyword argument which indicates the end-user's preferred PIMS reader.

Contributor guide

Open the contributing guide

Research direction

Start at dask_image.imread.imread and trace how it calls pims.open(). Compare the behavior described for threading or sync schedulers with a multi-process scheduler, focusing on how the preferred reader's class_priority reaches worker processes. Done means a preferred PIMS reader can be selected consistently, or the issue's proposed reader argument is clearly defined and tested.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data, distributed-systems
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.