scverse / scverse/spatialdata

get_extent(shapes, exact=True) (default) transforms every geometry even without rotation, and the exact=False path uses a Python-level .apply: 49 s vs ~0.2 s possible on 5.5 M Visium HD bins

Open
#1,218 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

element: shapes ▲ needs: triage performance 🚀 priority: medium
Dominant language
Python
Stars
394
Forks
95
Avg merge
4d 3h
Merged PRs (30d)
7

Description

[!NOTE]
This whole message is AI-generated. The issue was automatically discovered and reported by an AI agent (Claude) during an autonomous bug hunt on the spatialdata code base. It has not been verified or triaged by a human yet; the needs: triage label is set so that a maintainer can confirm it. The reproduction script below was executed by the agent in an isolated environment (see Environment) and its output is pasted verbatim.

Summary

On visium_hd_3.0.0_io (5.48 M polygons, pure scale transformation): get_extent(exact=True) 49.1 s / 9.7 GB peak, get_extent(exact=False) 5.0 s with the identical result; of those 5 s, 4.8 s are e["geometry"].apply(lambda geom: not geom.is_empty) while the vectorised ~e.geometry.is_empty takes 0.04 s and .bounds 0.1 s. The synthetic script below (2 M squares) shows the same ratios.

Severity (agent's assessment): medium/high for real data — get_extent is called by plotting and transform_to_data_extent

Where: src/spatialdata/_core/data_extent.py (GeoDataFrame overload of get_extent, _get_extent_of_shapes)

Expected behaviour

Bounds of millions of polygons in well under a second when the transformation has no rotation/shear.

Reproduction

Save as repro.py and run uv run repro.py (the PEP 723 header pins spatialdata to the commit the bug was found on; replace the URL fragment with @main to test the current main branch).

# /// script
# requires-python = ">=3.12"
# dependencies = [
#     "spatialdata @ git+https://github.com/scverse/spatialdata.git@ccf1ea048d054b6624214bf618008a9f9ae223e0",
# ]
# ///
"""get_extent(shapes) default (exact=True) transforms every geometry even for a pure scale, and the
exact=False path spends its time in a Python-level .apply over all geometries."""
import time
import warnings
import numpy as np
import shapely
import geopandas as gpd
from spatialdata import get_extent
from spatialdata.models import ShapesModel
from spatialdata.transformations import Scale

warnings.simplefilter("ignore")
n = 2_000_000
rng = np.random.default_rng(0)
xs, ys = rng.uniform(0, 5000, n), rng.uniform(0, 5000, n)
shapes = ShapesModel.parse(gpd.GeoDataFrame({"geometry": shapely.box(xs, ys, xs + 2, ys + 2)}), transformations={"global": Scale([0.5, 0.5], axes=("x", "y"))})
t0 = time.time(); e_exact = get_extent(shapes, exact=True); t_exact = time.time() - t0
t0 = time.time(); e_approx = get_extent(shapes, exact=False); t_approx = time.time() - t0
t0 = time.time(); shapes["geometry"].apply(lambda g: not g.is_empty); t_apply = time.time() - t0
t0 = time.time(); ~shapes.geometry.is_empty; t_vec = time.time() - t0
t0 = time.time(); shapes.geometry.bounds; t_bounds = time.time() - t0
print(f"{n:,} polygons, Scale transformation")
print(f"get_extent(exact=True)  {t_exact:6.1f}s  -> {e_exact}")
print(f"get_extent(exact=False) {t_approx:6.1f}s  -> {e_approx}   (same result)")
print(f"components: .apply(not is_empty) {t_apply:.1f}s | vectorised ~is_empty {t_vec:.3f}s | .bounds {t_bounds:.2f}s")
same = all(np.allclose(e_exact[k], e_approx[k]) for k in e_exact)
bug = same and t_exact > 3 * t_approx and t_apply > 10 * t_vec
print("VERDICT:", "BUG REPRODUCED (large avoidable cost)" if bug else "NOT REPRODUCED")
Observed output
2,000,000 polygons, Scale transformation
get_extent(exact=True)    18.3s  -> {'x': (np.float64(0.0014883595433889951), np.float64(2500.9997320480365)), 'y': (np.float64(0.002386817034849553), np.float64(2500.9985742560502))}
get_extent(exact=False)    4.5s  -> {'x': (np.float64(0.0014883595433889951), np.float64(2500.9997320480365)), 'y': (np.float64(0.002386817034849553), np.float64(2500.9985742560502))}   (same result)
components: .apply(not is_empty) 1.8s | vectorised ~is_empty 0.014s | .bounds 0.03s
VERDICT: BUG REPRODUCED (large avoidable cost)

Possible fix direction (unverified)

Take the exact=False path automatically when rotation and shear are identity (_decompose_transformation_full), and replace the .apply with the vectorised ~e.geometry.is_empty. The remaining cost of the exact path is the per-geometry transform() reported separately.

Environment

uv run repro.py with the PEP 723 metadata in the script (fresh, isolated environment; spatialdata built from main @ ccf1ea0 (2026-08-28); Python 3.13, latest releases of the dependencies at run time: pandas 3.0, anndata 0.13, zarr 3.3, dask 2026.8, numpy 2.5, geopandas 1.1, shapely 2.1). macOS (arm64). Also reproduced in a second environment with pandas 2.3.3 / anndata 0.12.11 / numpy 2.4.4 / zarr 3.2.1.


Automatically generated; discovered by an AI agent (Claude) and not yet reviewed by a human.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by running the provided repro.py with the pinned spatialdata commit, then read src/spatialdata/_core/data_extent.py, focusing on the GeoDataFrame overload of get_extent and _get_extent_of_shapes. Compare exact and non-exact paths for the pure Scale case and verify that the completed change preserves identical extents while avoiding the reported per-geometry and Python-level filtering costs.

Written by the indexing model from the issue text.

Assessment

Tech stack
numpy, pandas, python
Domain
data, performance
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
52/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.