scverse / scverse/spatialdata

Joins crash (IndexError / KeyError: None) when the element and the table share no instance

Open
#1,226 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug 🚨 element: table 📑 method: query needs: triage priority: medium
Dominant language
Python
Stars
394
Forks
95
Avg merge
4d 3h
Merged PRs (30d)
7

Description

[!NOTE]
This whole message is AI-generated. The issue was automatically discovered and reported by an AI agent (Claude) during an autonomous bug hunt on the spatialdata code base. It has not been verified or triaged by a human yet; the needs: triage label is set so that a maintainer can confirm it. The reproduction script below was executed by the agent in an isolated environment (see Environment) and its output is pasted verbatim.

Summary

Shapes with index [0, 1] and a table with instance ids [10, 11]: leftIndexError: list index out of range (all match_rows), inner/rightKeyError: None or IndexError: single positional indexer is out-of-bounds. left_exclusive/right_exclusive behave.

Severity (agent's assessment): medium — also reached through subset(), filter_by_coordinate_system() and spatial queries with filter_table=True (they use a left join) when a query drops all annotated instances

Where: src/spatialdata/_core/query/relational_query.py_region_as_str_if_list_of_len_one(region) does region[0] on an empty list; _get_masked_element does element.loc[None, :] when nothing matches; _match_rows does index_left.iloc[0] on an empty merge

Expected behaviour

An empty (0-row) table with valid metadata, or None, and an empty element — consistently across join types.

Reproduction

Save as repro.py and run uv run repro.py (the PEP 723 header pins spatialdata to the commit the bug was found on; replace the URL fragment with @main to test the current main branch).

# /// script
# requires-python = ">=3.12"
# dependencies = [
#     "spatialdata @ git+https://github.com/scverse/spatialdata.git@ccf1ea048d054b6624214bf618008a9f9ae223e0",
# ]
# ///
"""left/inner/right joins crash with IndexError/KeyError when the element and the table share no instance."""
import warnings
import numpy as np
import pandas as pd
import geopandas as gpd
from anndata import AnnData
from shapely.geometry import Point
from spatialdata import SpatialData, join_spatialelement_table
from spatialdata.models import ShapesModel, TableModel

warnings.simplefilter("ignore")
shapes = ShapesModel.parse(gpd.GeoDataFrame({"geometry": [Point(0, 0), Point(1, 1)], "radius": [1.0, 1.0]}, index=[0, 1]))
obs = pd.DataFrame({"region": pd.Categorical(["shp"] * 2), "instance_id": [10, 11]})  # no overlap with [0, 1]
table = TableModel.parse(AnnData(X=np.zeros((2, 1)), obs=obs), region="shp", region_key="region", instance_key="instance_id")
sdata = SpatialData(shapes={"shp": shapes}, tables={"t": table})
bug = False
for how in ["left", "inner", "right"]:
    for match_rows in ["no", "left", "right"]:
        try:
            elements, joined = join_spatialelement_table(sdata=sdata, spatial_element_names="shp", table_name="t", how=how, match_rows=match_rows)
            print(f"how={how:5s} match_rows={match_rows:5s}: OK (element rows={None if elements['shp'] is None else len(elements['shp'])}, table={None if joined is None else joined.shape})")
        except Exception as e:  # noqa: BLE001
            print(f"how={how:5s} match_rows={match_rows:5s}: {type(e).__name__}: {str(e)[:80]}")
            bug = True
print("VERDICT:", "BUG REPRODUCED" if bug else "NOT REPRODUCED")
Observed output
how=left  match_rows=no   : IndexError: list index out of range
how=left  match_rows=left : IndexError: list index out of range
how=left  match_rows=right: IndexError: list index out of range
how=inner match_rows=no   : KeyError: None
how=inner match_rows=left : IndexError: list index out of range
how=inner match_rows=right: IndexError: single positional indexer is out-of-bounds
how=right match_rows=no   : KeyError: None
how=right match_rows=left : KeyError: None
how=right match_rows=right: IndexError: single positional indexer is out-of-bounds
VERDICT: BUG REPRODUCED

Possible fix direction (unverified)

Guard the three spots above for the empty case.

Environment

uv run repro.py with the PEP 723 metadata in the script (fresh, isolated environment; spatialdata built from main @ ccf1ea0 (2026-08-28); Python 3.13, latest releases of the dependencies at run time: pandas 3.0, anndata 0.13, zarr 3.3, dask 2026.8, numpy 2.5, geopandas 1.1, shapely 2.1). macOS (arm64). Also reproduced in a second environment with pandas 2.3.3 / anndata 0.12.11 / numpy 2.4.4 / zarr 3.2.1.


Automatically generated; discovered by an AI agent (Claude) and not yet reviewed by a human.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by running the provided repro.py with uv, then inspect src/spatialdata/_core/query/relational_query.py, especially _region_as_str_if_list_of_len_one, _get_masked_element, and _match_rows. The fix is done when non-overlapping element and table instances no longer crash for the listed join types and match_rows values, and the empty-result behavior matches the expected None or empty table.

Written by the indexing model from the issue text.

Assessment

Tech stack
pandas, python
Domain
data
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
72/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.