Add a perception of a __xarray__ magic method

Is your feature request related to a problem?

I am often moving data from external objects (of all sorts!) into xarray. This is a common use case

Much of this code would be greatly simplified if there was a way of giving non-xarray classes a way of declaring to xarray how these objects can be marshaled into

Describe the solution you'd like

So here is an initial proposal for comment. Much of this could be implemented in a third party library. But doing this in xarray itself would likely be best.

Magic Methods

It would be great to see these magic method signatures become integrated throughout the library:

___xarray__ -> xr.Dataset | xr.DataArray 
___xarray_array__ -> xr.DatArray 
___xarray_dataset__ -> xr.Dataset
___xarray_datatree__ -> xr.DataTree   # when DataTree is finally integrated into xarray

Conversion Registry

And these extension functions to register converters:

def register_xarray_converter(class, name: str, func : Callable[[class, ...] | None) -> xr.Dataset | xr.DataArray]:
  ...
def register_dataarray_converter(class, name: str, func : Callable[[class, ...] | None) ->  xr.DataArray:
  ...
def register_dataset_converter(class, name: str,  func : Callable[[class, ...] | None) -> xr.Dataset:
  ...
def register_datatree_converter(class, name: str,  func : Callable[[class, ...], xr.DataArray] | None) -> DataTree # when DataTree is finally integrated into xarray
  ...

Registering a converter if if cls implements a corresponding xarray*_ method or another converter already registered for cls. Perhaps add an argument that specifies if the converter should or should not be added if their is a clash. Perhaps these functions return the replaced converter so it can be added back in if needed?

Ideally, also, "deregister" versions (.e.g deregister would also be available. So context managers that change marshaling behavior could easily be constructed.

User API

Along with the following new user API functions:

def as_xarray(x, *args, **kwargs) -> xr.Dataset | xr.DataArray:
  ...
def as_dataarray(x,*args, **kwargs) -> xr.DataArray:
  ...
def as_dataset(x,*args, **kwargs) -> xr.DataSet:
  ...
def as_dataset(x,*args, **kwargs) -> xr.DataSet: # when DataTree is finally integrated into xarray
  ...

"as_xarray" returns (in order of precedence:

• x unaltered if it is an xarray objects
• registered_xarray_converter(x, *args, **kwargs) if it is callable and does not throw an exception
• registered_dataarray_converter(x, *args, **kwargs) if it is callable and does not throw an exception
• registered_dataarray_converter(x, *args, **kwargs) if it is callable and does not throw an exception
• x.xarray(*args, **kwargs), if it exits, is callable, and does not throw an exception
• x.xarray_dataset(*args, **kwargs), if it exists, is callable, and does not throw an exception
• x.xarray_dataarray(*args, **kwargs), if it exists, is callable, and does not throw an exception
• well known aliases of xarray_dataarray, such as x.to_xarray(*args, **kwargs) (see pandas)
• [DESIGN DECISION] convert and return tuple[dims, data, [attr, encoding] to DataArray?
• [DESIGN DECISION] convert and return tuple encoding of DataSet?
• [DESIGN DECISION] return DataArray wrapped duck-typed array in DataArray?

The rationale for putting the registered functions first is that this would enable

"as_dataarrray" would be slimilar, but it would only call x.xarray_dataarray and well known aliases.

"as_dataset" would be slimilar, but it would only call x.xarray_dataset, well known aliases, and perhaps falling back to calling x.xarray_dataarray and converting the return a dataset if it has a name attribute.

"as_datatree" would be slimilar, but it would only call x.xarray_datatree, and perhaps falling back to calling x.xarray_dataarray and wrapping it in a single node datatree. (Though of course at this point this method would probably be implemented by the DataTree package, not xarray)

The design decisions are flexible from my point of view, and might be decided in a way that makes the code base simplest or most usable. There is also a question of whether or not this method should default the backup methods. These decisions also can be deferred entirely by delegating to the converter registry.

Across the Xarray Library

Finally, across the xarray library, there may be places where passing input arguments through as_xarray, as_dataarray, or as_dataset would make a lot of sense. This could be the final thing to do, but cannot be handled by a third party library.

Doing this would give give another pathway for third party libraries to integrate with xarray, with a far easier way than the converter registry or explicit calls to as_* functions.

Describe alternatives you've considered

This can be done with a private library. But it seems to a lot of code that is pretty useful to other use cases.

Most of this (but not all) can accomplished in a 3rd party library, but it wouldn't allow the seamless sort of integration with (for example) xarray use of repr_html to integrate with pandas.

The existing backend hooks work great when we are marshaling from file-based sources. See, for example, tiffslide-xarray (https://github.com/swamidasslab/tiffslide-xarray). This approach is seemless for reading files, but cannot marshal objects. For example, this is possible:

x = xr.open_dataset("slide.tiff")

But this doesn't work.

t = tiffslide.TiffSlide("slide.tiff")
x = xr.open_dataset(t) # won't work
x = xr.DataArray(t) # won't work either

This is an important use case because there are cases where we want to create an xarray like this from objects that are never stored on the filesystem.

Additional context

No response

Start by reviewing the existing backend hooks and the object-conversion examples involving xr.open_dataset, xr.DataArray, and pandas-style to_xarray methods. The proposal does not identify implementation files or tests; completion would require settling the magic-method, converter-registry, and as_* API design before integrating it across the library.