scverse / scverse/spatialdata

Joins on an element with duplicated index values duplicate rows (inner/right) instead of erroring

Open
#1,242 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug 🚨 element: shapes ▲ method: query needs: triage
Dominant language
Python
Stars
394
Forks
95
Avg merge
4d 3h
Merged PRs (30d)
7

Description

[!NOTE]
This whole message is AI-generated. The issue was automatically discovered and reported by an AI agent (Claude) during an autonomous bug hunt on the spatialdata code base. It has not been verified or triaged by a human yet; the needs: triage label is set so that a maintainer can confirm it. The reproduction script below was executed by the agent in an isolated environment (see Environment) and its output is pasted verbatim.

Summary

Shapes with index [0, 0, 1], table ids [0, 1]: inner/right joins return an element with index [0, 0, 0, 0, 1] (2 rows with id 0 became 4); left with match_rows="left" duplicates the table row for id 0.

Severity (agent's assessment): low/medium — the models do not forbid duplicated indices (they arise when concatenating per-ROI segmentations) and nothing warns

Where: src/spatialdata/_core/query/relational_query.py::_get_masked_element / _match_rows (element.loc[mask_values] with mask_values already containing the duplicates)

Expected behaviour

Either a clear error for non-unique element indices (validated in the models or at join time) or a boolean-mask selection that does not inflate rows.

Reproduction

Save as repro.py and run uv run repro.py (the PEP 723 header pins spatialdata to the commit the bug was found on; replace the URL fragment with @main to test the current main branch).

# /// script
# requires-python = ">=3.12"
# dependencies = [
#     "spatialdata @ git+https://github.com/scverse/spatialdata.git@ccf1ea048d054b6624214bf618008a9f9ae223e0",
# ]
# ///
"""Joins on an element with duplicated index values duplicate rows (inner/right) instead of erroring."""
import warnings
import numpy as np
import pandas as pd
import geopandas as gpd
from anndata import AnnData
from shapely.geometry import Point
from spatialdata import SpatialData, join_spatialelement_table
from spatialdata.models import ShapesModel, TableModel

warnings.simplefilter("ignore")
shapes = ShapesModel.parse(gpd.GeoDataFrame({"geometry": [Point(0, 0), Point(1, 1), Point(2, 2)], "radius": [1.0] * 3}, index=[0, 0, 1]))
obs = pd.DataFrame({"region": pd.Categorical(["shp"] * 2), "instance_id": [0, 1]})
table = TableModel.parse(AnnData(X=np.zeros((2, 1)), obs=obs), region="shp", region_key="region", instance_key="instance_id")
sdata = SpatialData(shapes={"shp": shapes}, tables={"t": table})
bug = False
for how, match_rows in [("left", "left"), ("inner", "no"), ("right", "right")]:
    elements, joined = join_spatialelement_table(sdata=sdata, spatial_element_names="shp", table_name="t", how=how, match_rows=match_rows)
    idx = elements["shp"].index.tolist()
    print(f"how={how:5s} match_rows={match_rows:5s}: element index {idx} (3 rows in input) | table ids {joined.obs.instance_id.tolist()}")
    bug |= len(idx) > 3
print("VERDICT:", "BUG REPRODUCED" if bug else "NOT REPRODUCED")
Observed output
how=left  match_rows=left : element index [0, 0, 1] (3 rows in input) | table ids [0, 0, 1]
how=inner match_rows=no   : element index [0, 0, 0, 0, 1] (3 rows in input) | table ids [0, 1]
how=right match_rows=right: element index [0, 0, 0, 0, 1] (3 rows in input) | table ids [0, 1]
VERDICT: BUG REPRODUCED

Possible fix direction (unverified)

Validate uniqueness in ShapesModel.validate/PointsModel.validate or at join time; select with element[element.index.isin(ids)].

Environment

uv run repro.py with the PEP 723 metadata in the script (fresh, isolated environment; spatialdata built from main @ ccf1ea0 (2026-08-28); Python 3.13, latest releases of the dependencies at run time: pandas 3.0, anndata 0.13, zarr 3.3, dask 2026.8, numpy 2.5, geopandas 1.1, shapely 2.1). macOS (arm64). Also reproduced in a second environment with pandas 2.3.3 / anndata 0.12.11 / numpy 2.4.4 / zarr 3.2.1.

Possibly related issues

#494


Automatically generated; discovered by an AI agent (Claude) and not yet reviewed by a human.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by running the provided repro with uv run repro.py, then inspect src/spatialdata/_core/query/relational_query.py, especially _get_masked_element and _match_rows. Confirm the duplicated-row behavior for inner and right joins, and determine whether the project should validate non-unique indices or use boolean-mask selection; done means the reproduction no longer inflates rows or produces the agreed clear error.

Written by the indexing model from the issue text.

Assessment

Tech stack
pandas, python
Domain
data
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
52/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.