scverse / scverse/spatialdata

Transform dimensions are changed when writing to zarr store

Open
#1,029 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

needs: triage priority: medium
Dominant language
Python
Stars
394
Forks
95
Avg merge
4d 3h
Merged PRs (30d)
7

Description

Hey everyone,

When I use a scale transform and write/read my data to a zarr store, the dimensions of the scale transform are changed for the image only. That breaks followup aggregations, as the dimensions between the transforms mismatch.

import spatialdata as sd
from spatialdata.datasets import blobs
from spatialdata.transformations import get_transformation, set_transformation, Scale

sdata = blobs()

scale_transform = Scale([0.1, 0.1], ("x", "y"))
set_transformation(sdata["blobs_image"], transformation=scale_transform, to_coordinate_system="global")
set_transformation(sdata["blobs_labels"], transformation=scale_transform, to_coordinate_system="global")

# Initial aggregation works
_ = sdata.aggregate(values="blobs_image", by="blobs_labels", target_coordinate_system="global")

# Write to zarr and read back (as for example recommended for better performance for visualization)
sdata.write("blobs.zarr")
sdata_zarr = sd.read_zarr("blobs.zarr")

# Aggregate no longer works
try:
    _ = sdata_zarr.aggregate(values="blobs_image", by="blobs_labels", target_coordinate_system="global")
except Exception as e:
    print(f"Error: {e}")
# Error: operands could not be broadcast together with shapes (2,) (3,) 

# This is due to a new first dimension added to the scale transform for the image only, resulting in 3 vs 2 dimensions
transform_image = get_transformation(sdata_zarr["blobs_image"], get_all=True)
print(transform_image)
# {'global': Scale (c, y, x)
#    [1.  0.1 0.1]}

transform_labels = get_transformation(sdata_zarr["blobs_labels"], get_all=True)
print(transform_labels)
#{'global': Scale (y, x)
#    [0.1 0.1]}

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by running the provided reproduction with Scale, sdata.write(), sd.read_zarr(), and aggregate(). Trace how the image and labels transformations are serialized and restored, especially the added image dimension. Done means both transformations retain matching dimensions after the round trip and the post-read aggregation succeeds.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
68/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.