scverse / scverse/spatialdata

Can I speed up the spatialdata.polygon_query when dealing with really big datasets?

Open
#893 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

method: query performance 🚀
Dominant language
Python
Stars
394
Forks
95
Avg merge
4d 3h
Merged PRs (30d)
7

Description

Hello,

thank you for providing the community with such a great tool!

I was wondering if you could help or advise on the following.

I have a very large xenium dataset (~2 million cells). I have read it and saved it as a zarr file and I have read it back in on my notebook. At the same time I have used QuPath to create annotations for my images, I have saved them as geojson files and read them back in as a geodataframes and subsequently converted them into polygons.

I would like to add this type of metadata information in my sdata object, so that I can annotate and filter my sdata.table based on that annotation. The way that I am currently trying to do this is by using the following piece of code, really in an effort to create subsets of my dataset based on polygons which I can then assign the annotation.

from spatialdata import polygon_query # type: ignore

cropped_sdata2 = polygon_query(
    sdata,
    polygon=polygon,
    target_coordinate_system="global"
)

This piece of code is extremely slow because of the size of the dataset, so I was wondering if there is a way to further speed up this function (I am already running this analysis on a server), or if there is a simpler way incorporate my annotation to the sdata object.

Thanks a lot for your help!

Cheers,
Anastasia

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start at the public spatialdata.polygon_query entry point and reproduce the reported slowdown with a large Xenium dataset, a Zarr-backed sdata object, and the GeoJSON-derived polygon. Measure the current query behavior and identify a concrete, testable performance target or simpler annotation workflow; the issue does not name implementation files or tests.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data, performance
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.