scverse / scverse/spatialdata

Interpolation when resampling images

Open
#166 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
394
Forks
95
Avg merge
4d 3h
Merged PRs (30d)
7

Description

When applying a transformation to an image element or a labels element, or when rasterizing an image or labels, dask_image.ndinterp.affine_transform() is called (you can search this function to find the two occurrences in the code).

At the moment we use the following or equivalent code:

    if schema == Labels2DModel or schema == Labels3DModel:
        kwargs = {"prefilter": False, "order": 0}
    elif schema == Image2DModel or schema == Image3DModel:
        kwargs = {}
    else:
        raise ValueError(f"Unsupported schema {schema}")

to ensure that no interpolation is performed for labels, the result can be seen for instance here https://github.com/scverse/spatialdata/issues/165.

For images interpolation happens, an example of this is in the screenshot example. We may want to do one of the following:

  1. expose kwargs to the user both for rasterize() and for BaseTransformation.transform()
  2. don't do anything
  3. use the same approach as for labels -> no interpolation, no antialiasing
  • ...

When this is done we should update the test test_rasterize_raster(), which at the moment can't perform a pixel to pixel comparison (also because of this other issue https://github.com/scverse/spatialdata/issues/165).

Screenshot example.

This
image

becomes this (the offset is due to https://github.com/scverse/spatialdata/issues/165)
image

when doing a rasterization with the parameter target_unit_to_pixels = 2

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Locate the two calls to dask_image.ndinterp.affine_transform() and inspect test_rasterize_raster(). First determine which interpolation policy is intended for images, including whether kwargs should be exposed or interpolation disabled; done means the chosen behavior is implemented for rasterize() and BaseTransformation.transform() and the test can perform a pixel-to-pixel comparison.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
computer-vision
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.