deepcopy() of a multiscale raster loads the original element into memory and detaches it from its Zarr backing
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 394
- Forks
- 95
- Avg merge
- 4d 3h
- Merged PRs (30d)
- 7
Description
[!NOTE]
This whole message is AI-generated. The issue was automatically discovered and reported by an AI agent (Claude) during an autonomous bug hunt on thespatialdatacode base. It has not been verified or triaged by a human yet; theneeds: triagelabel is set so that a maintainer can confirm it. The reproduction script below was executed by the agent in an isolated environment (see Environment) and its output is pasted verbatim.
Summary
After deepcopy(sdata) every multiscale image/labels of sdata is an in-memory numpy-backed dask array (dask key changes from original-from-zarr-... to array-..., backing files go from 1 to 0). The DataArray overload does not have this problem.
Severity (agent's assessment): medium/high — a copy operation changes the source: 2× memory, the original is no longer lazy, get_dask_backing_files() returns nothing
Where: src/spatialdata/_core/_deepcopy.py, DataTree overload (element[key][variable] = element[key][variable].compute() and element[key][variable].data = from_array(...) on the input)
Expected behaviour
The input is left untouched.
Reproduction
Save as repro.py and run uv run repro.py (the PEP 723 header pins spatialdata to the commit the bug was found on; replace the URL fragment with @main to test the current main branch).
# /// script
# requires-python = ">=3.12"
# dependencies = [
# "spatialdata @ git+https://github.com/scverse/spatialdata.git@ccf1ea048d054b6624214bf618008a9f9ae223e0",
# ]
# ///
"""deepcopy() of a multiscale raster loads the ORIGINAL element into memory and detaches it from its Zarr backing."""
import os
import shutil
import tempfile
import warnings
import numpy as np
from spatialdata import SpatialData, deepcopy, get_dask_backing_files, read_zarr
from spatialdata.models import Image2DModel
warnings.simplefilter("ignore")
tmp = tempfile.mkdtemp()
img = Image2DModel.parse(np.random.default_rng(0).integers(0, 255, (3, 64, 64), dtype=np.uint8), scale_factors=[2, 2])
SpatialData(images={"img": img}).write(os.path.join(tmp, "store.zarr"))
sdata = read_zarr(os.path.join(tmp, "store.zarr"))
original = sdata["img"]
key_before = str(next(iter(original["scale0"]["image"].data.dask.keys())))[:40]
files_before = len(get_dask_backing_files(original))
copy = deepcopy(sdata)
key_after = str(next(iter(original["scale0"]["image"].data.dask.keys())))[:40]
files_after = len(get_dask_backing_files(original))
print(f"original before deepcopy: dask key {key_before!r}, backing files {files_before}")
print(f"original after deepcopy: dask key {key_after!r}, backing files {files_after} (expected: unchanged, still zarr-backed)")
shutil.rmtree(tmp)
bug = files_before > 0 and files_after == 0
print("VERDICT:", "BUG REPRODUCED (input mutated)" if bug else "NOT REPRODUCED")
Observed output
original before deepcopy: dask key 'original-from-zarr-552b983f3249439bc3c18', backing files 1
original after deepcopy: dask key "('array-dfe5545938945c60891068d78394d540", backing files 0 (expected: unchanged, still zarr-backed)
VERDICT: BUG REPRODUCED (input mutated)
Possible fix direction (unverified)
Compute into the new tree only (build the copied DataTree from computed arrays) and never assign back into element.
Environment
uv run repro.py with the PEP 723 metadata in the script (fresh, isolated environment; spatialdata built from main @ ccf1ea0 (2026-08-28); Python 3.13, latest releases of the dependencies at run time: pandas 3.0, anndata 0.13, zarr 3.3, dask 2026.8, numpy 2.5, geopandas 1.1, shapely 2.1). macOS (arm64). Also reproduced in a second environment with pandas 2.3.3 / anndata 0.12.11 / numpy 2.4.4 / zarr 3.2.1.
Possibly related issues
#286
Automatically generated; discovered by an AI agent (Claude) and not yet reviewed by a human.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in src/spatialdata/_core/_deepcopy.py, focusing on the DataTree overload and its handling of multiscale elements. Run the provided repro.py with uv and inspect how the copied tree and original element are changed. Done means deepcopy leaves the original multiscale raster lazy and Zarr-backed, with its dask key and backing-file count unchanged.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- data
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 72/100