scverse / scverse/spatialdata

Adding `.pipe` to `SpatialData`

Open
#695 6 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement ✨
Dominant language
Python
Stars
394
Forks
95
Avg merge
4d 3h
Merged PRs (30d)
7

Description

Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]

It'd be elegant to be able to chain functions on a SpatialData object. Currently given some functions f,g,h

def f(sdata: sd.SpatialData, *) -> sd.SpatialData: ...
def g(sdata: sd.SpatialData, arg1: Any, arg2: Any, *) -> sd.SpatialData: ...
def h(arg3: Any, sdata: sd.SpatialData, *) -> sd.SpatialData: ...

We would have to do the following:

sdata_h = h(arg3=c, sdata=g(f(sdata), arg1=a, arg2=b))

# or

sdata_h = h(sdata)
sdata_g = g(sdata_f, arg1=a, arg2=b)
sdata_f = f(arg3=c, sdata=sdata_h)

Describe the solution you'd like

Pandas and Xarray have pipe methods for DataFrames, DataArrays and Datasets, looking over their examples the pipe here would be able to be used like so:

sdata.pipe(f, arg1=a).pipe(g, arg2=b).pipe((h, "sdata"), arg3=c)

Describe alternatives you've considered

If a user has their own custom SpatialData Accessors for f,g,h (where h's first argument is a SpatialData / self object in this case), then it should work just the same, but incorporating the accessor call within a lambda function makes it rather wordy.


sdata = (
    sdata.pipe(lambda s: s.my_accessor.f())
    .pipe(lambda s: s.my_accessor.g(arg1=a, arg2=b))
    .pipe(lambda s: s.my_accessor.h(arg3=c))
)

Just chaining the accessor is much easier to read in this instance.

sdata.my_accessor.f().myaccessor.g(arg1=a, arg2=b).my_accessor.h(arg3=c)

For accessors, piping would be more useful in contexts where there are higher order functions composed of calls to the accessor's methods:

def f(sdata: sd.SpatialData, arg1, arg2) -> sd.SpatialData:
	intermediate_sdata = sdata.my_accessor.h(arg1).my_accessor.g(arg2)
	something_has_been_done = do_something_else(intermediate_sdata)
	return something_has_been_done

def i(sdata: sd.SpatialData, arg3) -> sd.SpatialData:
	intermediate_sdata = sdata.my_accessor.h(arg3)
	something_has_been_done2 = do_something_else2(intermediate_sdata)
	return something_has_been_done2

modified_sdata = sdata.pipe(f, arg1=a, arg2=b).pipe(i, arg3=c)

Additional context

Implementation:
The following has been taken from https://github.com/pydata/xarray/blob/d33e4ad9407591cc7287973b0f8da47cae396004/xarray/core/common.py#L717-L847

P = ParamSpec("P")
T = TypeVar("T")

class SpatialData:
    ...
    def pipe(self, func: Callable[P, T] | tuple[Callable[P, T], str], *args: P.args, **kwargs: P.kwargs) -> Any:
        if isinstance(func, tuple):
            func, target = func
            if target in kwargs:
                raise ValueError(f"{target} is both the pipe target and a keyword argument")
            kwargs[target] = self
            return func(*args, **kwargs)
        else:
            return func(self, *args, **kwargs)

These pipes can return anything so users would have to keep that in mind if they plan on chaining multiple calls to pipe.

References:

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by locating the SpatialData class and reviewing the referenced xarray pipe implementation. The feature is done when pipe supports direct function calls and tuple-target calls with the documented argument behavior, with tests covering both forms.

Written by the indexing model from the issue text.

Assessment

Tech stack
pandas, python
Domain
api
Issue type
Feature
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.