Provide protocols for creating structural subtypes of DataArray/Dataset
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 4.2k
- Forks
- 1.4k
- Avg merge
- 2d 15h
- Merged PRs (30d)
- 14
Description
Is your feature request related to a problem?
I frequently find myself wanting to annotate functions in terms of xarray objects that adhere to a particular schema. Given that a dataset's adherence to a schema is a matter of its structure/contents, it is unnatural to try to describe a schema as a subtype of xr.Dataset (or DataArray) (i.e. a type-checker ought not care that a dataset is an instance of a specific subclass of Dataset).
Describe the solution you'd like
Instead, it would be ideal to define a schema as a Protocol (structural subtype) of xr.Dataset. Unfortunately, one cannot subclass a normal class to create a protocol.
Thus, I am proposing that xarray provide Protocol-based descriptions of DataArray and Dataset so that users can describe schemas as structural subtypes of these classes. E.g.
from typing import Protocol
from xarray import DataArray
from xarray.typing import DatasetProtocol
class ClimateData(DatasetProtocol, Protocol):
lat: DataArray
lon: DataArray
temp: DataArray
precip: DataArray
def process_climate_data(ds: ClimateData):
ds.banana # type checker flags as unknown attribute
ds.temp # type checker sees "DataArray" (as informed by ClimateData)
ds.sel(lat=1.0) # type checker sees `Dataset` (as informed by `DatasetProtocol`)
The contents of DatasetProtocol would essentially look like a modified type stub for xarray.Dataset so the implementation details are relatively simple, I believe.
Describe alternatives you've considered
Creating a strict subtype of Dataset is not ideal for a few reasons:
- Static type checkers would then expect to see that datasets must derive from that particular subclass, which is generally not the case.
- The annotations / design of
xarray.Datasetis too broad for describing a schema. E.g. the presence of__getattr__prevents type checkers from flagging access to non-existent data variables and coordinates during static analysis.DatasetProtocolwould need to be designed to be less permissive than this.
Additional context
Hopefully this could be leveraged by the likes of xarray-schema so that xarray schemas can be used to provide both runtime and static validation capabilities.
I'd love to get feedback on this, and would be happy to open a PR if xarray devs are willing to weigh in on the design of these protocols.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the existing xarray typing surface and the Dataset/DataArray annotations, then review PEP 544 and the proposed xarray.typing.DatasetProtocol example. Define what the protocols must expose and how schema protocols should narrow dataset attributes while retaining Dataset methods. Done means users can declare structural schema subtypes and type checkers recognize known attributes while rejecting unknown ones.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- data, developer-experience
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100