scverse / scverse/spatialdata

Adding image to same slot for backed sdata

Open
#374 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
394
Forks
95
Avg merge
4d 3h
Merged PRs (30d)
7

Description

When writing an image to zarr with overwrite=True, I found some unexpected behaviour.
Below I give a mimimal example to reproduce the issue.

This works, as expected. I can overwrite image layer 'image'.

import spatialdata
import numpy as np

sdata = spatialdata.SpatialData()

sdata.write( 'sdata.zarr' )

arr=np.random.rand( 1,100,100 )

spatialelement=spatialdata.models.Image2DModel.parse(
            arr, dims=[ 'c' ,'y', 'x' ],
        )

sdata.add_image( name="image", image=spatialelement , overwrite=True )

sdata.add_image( name="image", image=spatialelement , overwrite=True )

However, when I add some processing (e.g. multiplying by two) before replacing sdata["image" ]:

spatialelement=spatialdata.models.Image2DModel.parse(
            sdata[ "image" ].data*2, dims=[ 'c' ,'y', 'x' ],
        )

sdata.add_image( name="image", image=spatialelement , overwrite=True )

I get the error:

ValueError Traceback (most recent call last)
in SpatialData.add_image(self, name, image, storage_options, overwrite)
659 target_path = os.path.realpath(os.path.join(self.path, "images", name))
660 if target_path in files:
--> 661 raise ValueError(
662 "Cannot add the image to the SpatialData object because it would overwrite an element that it is"
663 "using for backing. See more here: https://github.com/scverse/spatialdata/pull/138"
664 )
665 self._add_image_in_memory(name=name, image=image, overwrite=overwrite)
666 # old code to support overwriting the backing file
667 # with tempfile.TemporaryDirectory() as tmpdir:
668 # store = parse_url(Path(tmpdir) / "data.zarr", mode="w").store
(...)
692 # image = _read_multiscale(str(tgt_element_path), raster_type="image")
693 # self._add_image_in_memory(name=name, image=image, overwrite=True)

ValueError: Cannot add the image to the SpatialData object because it would overwrite an element that it isusing for backing. See more here: https://github.com/scverse/spatialdata/pull/138

However, if I do a persist(), I am allowed to overwrite:

spatialelement=spatialdata.models.Image2DModel.parse(
            (sdata[ "image" ].data*2).persist(), dims=[ 'c' ,'y', 'x' ],
        )

sdata.add_image( name="image", image=spatialelement , overwrite=True )

And I am also allowed to do this (typical scenario where you want to do some processing twice with e.g. different parameters etc.):

spatialelement=spatialdata.models.Image2DModel.parse(
            sdata[ "image" ].data*2, dims=[ 'c' ,'y', 'x' ],
        )

sdata.add_image( name="image2", image=spatialelement , overwrite=True )

spatialelement=spatialdata.models.Image2DModel.parse(
            sdata[ "image" ].data*2, dims=[ 'c' ,'y', 'x' ],
        )

sdata.add_image( name="image2", image=spatialelement , overwrite=True )

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start at SpatialData.add_image, especially the backing-file check around the reported traceback, and reproduce the issue with the minimal examples in the report. Compare lazy data from sdata["image"].data with the persisted result and the image2 case. Done means the intended overwrite behavior is established and covered by a regression test.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.