Adding image to same slot for backed sdata
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 394
- Forks
- 95
- Avg merge
- 4d 3h
- Merged PRs (30d)
- 7
Description
When writing an image to zarr with overwrite=True, I found some unexpected behaviour.
Below I give a mimimal example to reproduce the issue.
This works, as expected. I can overwrite image layer 'image'.
import spatialdata
import numpy as np
sdata = spatialdata.SpatialData()
sdata.write( 'sdata.zarr' )
arr=np.random.rand( 1,100,100 )
spatialelement=spatialdata.models.Image2DModel.parse(
arr, dims=[ 'c' ,'y', 'x' ],
)
sdata.add_image( name="image", image=spatialelement , overwrite=True )
sdata.add_image( name="image", image=spatialelement , overwrite=True )
However, when I add some processing (e.g. multiplying by two) before replacing sdata["image" ]:
spatialelement=spatialdata.models.Image2DModel.parse(
sdata[ "image" ].data*2, dims=[ 'c' ,'y', 'x' ],
)
sdata.add_image( name="image", image=spatialelement , overwrite=True )
I get the error:
ValueError Traceback (most recent call last)
in SpatialData.add_image(self, name, image, storage_options, overwrite)
659 target_path = os.path.realpath(os.path.join(self.path, "images", name))
660 if target_path in files:
--> 661 raise ValueError(
662 "Cannot add the image to the SpatialData object because it would overwrite an element that it is"
663 "using for backing. See more here: https://github.com/scverse/spatialdata/pull/138"
664 )
665 self._add_image_in_memory(name=name, image=image, overwrite=overwrite)
666 # old code to support overwriting the backing file
667 # with tempfile.TemporaryDirectory() as tmpdir:
668 # store = parse_url(Path(tmpdir) / "data.zarr", mode="w").store
(...)
692 # image = _read_multiscale(str(tgt_element_path), raster_type="image")
693 # self._add_image_in_memory(name=name, image=image, overwrite=True)ValueError: Cannot add the image to the SpatialData object because it would overwrite an element that it isusing for backing. See more here: https://github.com/scverse/spatialdata/pull/138
However, if I do a persist(), I am allowed to overwrite:
spatialelement=spatialdata.models.Image2DModel.parse(
(sdata[ "image" ].data*2).persist(), dims=[ 'c' ,'y', 'x' ],
)
sdata.add_image( name="image", image=spatialelement , overwrite=True )
And I am also allowed to do this (typical scenario where you want to do some processing twice with e.g. different parameters etc.):
spatialelement=spatialdata.models.Image2DModel.parse(
sdata[ "image" ].data*2, dims=[ 'c' ,'y', 'x' ],
)
sdata.add_image( name="image2", image=spatialelement , overwrite=True )
spatialelement=spatialdata.models.Image2DModel.parse(
sdata[ "image" ].data*2, dims=[ 'c' ,'y', 'x' ],
)
sdata.add_image( name="image2", image=spatialelement , overwrite=True )
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start at SpatialData.add_image, especially the backing-file check around the reported traceback, and reproduce the issue with the minimal examples in the report. Compare lazy data from sdata["image"].data with the persisted result and the image2 case. Done means the intended overwrite behavior is established and covered by a regression test.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- data
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100