scverse / scverse/spatialdata

deepcopy() of a multiscale raster loads the original element into memory and detaches it from its Zarr backing

Open
#1,223 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug 🚨 element: images 🌌 element: labels 🏷️ needs: triage priority: medium
Dominant language
Python
Stars
394
Forks
95
Avg merge
4d 3h
Merged PRs (30d)
7

Description

[!NOTE]
This whole message is AI-generated. The issue was automatically discovered and reported by an AI agent (Claude) during an autonomous bug hunt on the spatialdata code base. It has not been verified or triaged by a human yet; the needs: triage label is set so that a maintainer can confirm it. The reproduction script below was executed by the agent in an isolated environment (see Environment) and its output is pasted verbatim.

Summary

After deepcopy(sdata) every multiscale image/labels of sdata is an in-memory numpy-backed dask array (dask key changes from original-from-zarr-... to array-..., backing files go from 1 to 0). The DataArray overload does not have this problem.

Severity (agent's assessment): medium/high — a copy operation changes the source: 2× memory, the original is no longer lazy, get_dask_backing_files() returns nothing

Where: src/spatialdata/_core/_deepcopy.py, DataTree overload (element[key][variable] = element[key][variable].compute() and element[key][variable].data = from_array(...) on the input)

Expected behaviour

The input is left untouched.

Reproduction

Save as repro.py and run uv run repro.py (the PEP 723 header pins spatialdata to the commit the bug was found on; replace the URL fragment with @main to test the current main branch).

# /// script
# requires-python = ">=3.12"
# dependencies = [
#     "spatialdata @ git+https://github.com/scverse/spatialdata.git@ccf1ea048d054b6624214bf618008a9f9ae223e0",
# ]
# ///
"""deepcopy() of a multiscale raster loads the ORIGINAL element into memory and detaches it from its Zarr backing."""
import os
import shutil
import tempfile
import warnings
import numpy as np
from spatialdata import SpatialData, deepcopy, get_dask_backing_files, read_zarr
from spatialdata.models import Image2DModel

warnings.simplefilter("ignore")
tmp = tempfile.mkdtemp()
img = Image2DModel.parse(np.random.default_rng(0).integers(0, 255, (3, 64, 64), dtype=np.uint8), scale_factors=[2, 2])
SpatialData(images={"img": img}).write(os.path.join(tmp, "store.zarr"))
sdata = read_zarr(os.path.join(tmp, "store.zarr"))
original = sdata["img"]
key_before = str(next(iter(original["scale0"]["image"].data.dask.keys())))[:40]
files_before = len(get_dask_backing_files(original))
copy = deepcopy(sdata)
key_after = str(next(iter(original["scale0"]["image"].data.dask.keys())))[:40]
files_after = len(get_dask_backing_files(original))
print(f"original before deepcopy: dask key {key_before!r}, backing files {files_before}")
print(f"original after  deepcopy: dask key {key_after!r}, backing files {files_after}   (expected: unchanged, still zarr-backed)")
shutil.rmtree(tmp)
bug = files_before > 0 and files_after == 0
print("VERDICT:", "BUG REPRODUCED (input mutated)" if bug else "NOT REPRODUCED")
Observed output
original before deepcopy: dask key 'original-from-zarr-552b983f3249439bc3c18', backing files 1
original after  deepcopy: dask key "('array-dfe5545938945c60891068d78394d540", backing files 0   (expected: unchanged, still zarr-backed)
VERDICT: BUG REPRODUCED (input mutated)

Possible fix direction (unverified)

Compute into the new tree only (build the copied DataTree from computed arrays) and never assign back into element.

Environment

uv run repro.py with the PEP 723 metadata in the script (fresh, isolated environment; spatialdata built from main @ ccf1ea0 (2026-08-28); Python 3.13, latest releases of the dependencies at run time: pandas 3.0, anndata 0.13, zarr 3.3, dask 2026.8, numpy 2.5, geopandas 1.1, shapely 2.1). macOS (arm64). Also reproduced in a second environment with pandas 2.3.3 / anndata 0.12.11 / numpy 2.4.4 / zarr 3.2.1.

Possibly related issues

#286


Automatically generated; discovered by an AI agent (Claude) and not yet reviewed by a human.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in src/spatialdata/_core/_deepcopy.py, focusing on the DataTree overload and its handling of multiscale elements. Run the provided repro.py with uv and inspect how the copied tree and original element are changed. Done means deepcopy leaves the original multiscale raster lazy and Zarr-backed, with its dask key and backing-file count unchanged.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
72/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.