integration of marray into xarray
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 4.2k
- Forks
- 1.4k
- Avg merge
- 2d 15h
- Merged PRs (30d)
- 14
Description
What is your issue?
During the scipy sprints this year I had a quick look at which parts of xarray already work with marray, via xarray's array API support. In following you'll see what I tried already.
Preamble for the examples below:
import xarray as xr
import marray
import numpy as np
xr.set_options(display_expand_data=False)
rng = np.random.default_rng()
What works already:
- creating xarray objects containing masked arrays:
mnp = marray.masked_namespace(np)
data = rng.normal(size=(2000, 1000)) * 2 - 1
masked = mnp.asarray(data, mask=np.abs(data) < 0.5)
arr = xr.DataArray(
masked,
dims=("time", "x"),
coords={"time": xr.date_range("2025-07-07 08:00:00", freq="6h", periods=2000), "x": np.arange(1000)},
)
- aggregation (without nan-skipping,
marraydoes that by default but doesn't implement__array_function__fornan*):arr.mean(dim="x", skipna=False) - subsetting:
arr.sel(time="2025-07-09") - reindexing:
arr.reindex(x=np.arange(-5, 15), fill_value=mnp.asarray(0, mask=True))(this wraps the data usingmarray, so won't work withdask. It does allow converting non-marraytomarraydata, though) - where:
arr.where(xr.ufuncs.abs(arr) > 2.4, mnp.asarray(0, mask=True)) - groupby aggregations:
arr.groupby("time.day").mean(skipna=False) - stack:
arr.stack(z=("time", "x")) - roll:
arr.roll({"time": 3})
What does not work yet:
- sortby:
arr.sortby("time")(fails with a "cannot pickle module object") - pad / shift:
arr.pad({"x": 2}, constant_values=mnp.asarray(0, mask=True))(padis not part of the array API, yet) - isnull:
arr.isnull()(it's not quite clear whether the name refers tonan/nat/None, or to truly missing values) - na-filling methods like
ffill/bfill/fillnaandinterpolate_na(we might need amaskedaccessor for that if we don't want to special-casemarray) - string methods (only works on numpy arrays? But since the vlen string dtype in numpy 2 supports missing values it might not be needed?)
To make a lot of these a bit less of a mouthful we could make the default placeholder for missing values (NA) be aware of marray and represent a masked 0d marray, or add a missing global to marray's namespace such that we could pass e.g. mnp.missing to all places where fill_value crops up.
cc @dcherian, @mdhaber
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with xarray's array API support and reproduce the listed marray examples, especially sortby, pad/shift, isnull, and the missing-value methods. The issue does not name files or tests or settle whether to add marray-aware missing values, an accessor, or broader support, so completion would need to be defined before implementation.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- numpy, python
- Domain
- data
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100