join_spatialelement_table crashes when obs index name matches an existing column
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 394
- Forks
- 95
- Avg merge
- 4d 3h
- Merged PRs (30d)
- 7
Description
Bug description
_inner_join_spatialelement_table and _left_join_spatialelement_table in spatialdata/_core/query/relational_query.py call table.obs.reset_index() (lines 390 and 471) without handling the case where the obs index name already exists as a column. This raises:
ValueError: cannot insert EntityID, already exists
How it manifests
In spatialdata-plot, render_shapes(color=...) calls join_spatialelement_table(..., how="inner"), which hits this crash. Users with Merfish data are affected because their tables have EntityID as both the obs index name and an obs column — a state that spatialdata's own validation allows.
Reported in https://github.com/scverse/spatialdata-plot/issues/441.
Minimal reproduction
import pandas as pd
from anndata import AnnData
from spatialdata.models import TableModel
obs = pd.DataFrame({
"region": pd.Categorical(["shapes"] * 5),
"EntityID": [0, 1, 2, 3, 4],
"cell_type": ["A", "B", "C", "A", "B"],
})
table = AnnData(obs=obs)
table = TableModel.parse(table, region="shapes", region_key="region", instance_key="EntityID")
# Simulate the state found in real Merfish data loaded from disk
table.obs.index = pd.Index([0, 1, 2, 3, 4], name="EntityID")
# This is what join_spatialelement_table does internally — crashes here:
table.obs.reset_index()
# ValueError: cannot insert EntityID, already exists
Suggested fix
In _inner_join_spatialelement_table (line 390) and _left_join_spatialelement_table (line 471), handle the collision before calling reset_index(). For example, drop the index name when it already exists as a column:
obs = table.obs
if obs.index.name is not None and obs.index.name in obs.columns:
obs = obs.reset_index(drop=True)
else:
obs = obs.reset_index()
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Read spatialdata/_core/query/relational_query.py, focusing on _inner_join_spatialelement_table around line 390 and _left_join_spatialelement_table around line 471. Run the minimal AnnData reproduction with an obs index named EntityID and verify that both inner and left joins complete without the reset_index collision.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- pandas, python
- Domain
- data
- Issue type
- Bug
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 84/100