DataArray.resample().apply() fails to apply custom function
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 4.2k
- Forks
- 1.4k
- Avg merge
- 2d 15h
- Merged PRs (30d)
- 14
Description
What happened?
Hi,
I try to apply a custom function to aggregate a resampled object via .apply().
Maybe it is a documentation issue? But I couldn't find it.
For example calculating the median by passing np.median fails with an error as shown in the log below.
What did you expect to happen?
I would expect the median or any other custom function to be calculated for the resampled data.
It seems to work with pure pandas.
Minimal Complete Verifiable Example
import numpy as np
import pandas as pd
import xarray as xr
idx = pd.date_range("2000-01-01", "2000-12-31")
data = xr.DataArray(np.random.randn(len(idx)), coords={"index": idx})
data.resample(index="M").apply(np.median)
MVCE confirmation
- Minimal example — the example is as focused as reasonably possible to demonstrate the underlying issue in xarray.
- Complete example — the example is self-contained, including all data and the text of any traceback.
- Verifiable example — the example copy & pastes into an IPython prompt or Binder notebook, returning the result.
- New issue — a search of GitHub Issues suggests this is not a duplicate.
Relevant log output
---------------------------------------------------------------------------
AttributeError Traceback (most recent call last)
Input In [425], in <cell line: 3>()
1 idx = pd.date_range("2000-01-01", "2000-12-31")
2 data = pd.Series(np.random.randn(len(idx)), index=idx).to_xarray()
----> 3 data.resample(index="M", label="left", loffset="15d").apply(np.median)
File ~/Work/miniconda3/envs/20211123_py39/lib/python3.9/site-packages/xarray/core/resample.py:279, in DataArrayResample.apply(self, func, args, shortcut, **kwargs)
267 """
268 Backward compatible implementation of ``map``
269
(...)
272 DataArrayResample.map
273 """
274 warnings.warn(
275 "Resample.apply may be deprecated in the future. Using Resample.map is encouraged",
276 PendingDeprecationWarning,
277 stacklevel=2,
278 )
--> 279 return self.map(func=func, shortcut=shortcut, args=args, **kwargs)
File ~/Work/miniconda3/envs/20211123_py39/lib/python3.9/site-packages/xarray/core/resample.py:253, in DataArrayResample.map(self, func, args, shortcut, **kwargs)
210 """Apply a function to each array in the group and concatenate them
211 together into a new array.
212
(...)
249 The result of splitting, applying and combining this array.
250 """
251 # TODO: the argument order for Resample doesn't match that for its parent,
252 # GroupBy
--> 253 combined = super().map(func, shortcut=shortcut, args=args, **kwargs)
255 # If the aggregation function didn't drop the original resampling
256 # dimension, then we need to do so before we can rename the proxy
257 # dimension we used.
258 if self._dim in combined.coords:
File ~/Work/miniconda3/envs/20211123_py39/lib/python3.9/site-packages/xarray/core/groupby.py:1095, in DataArrayGroupByBase.map(self, func, args, shortcut, **kwargs)
1093 grouped = self._iter_grouped_shortcut() if shortcut else self._iter_grouped()
1094 applied = (maybe_wrap_array(arr, func(arr, *args, **kwargs)) for arr in grouped)
-> 1095 return self._combine(applied, shortcut=shortcut)
File ~/Work/miniconda3/envs/20211123_py39/lib/python3.9/site-packages/xarray/core/groupby.py:1115, in DataArrayGroupByBase._combine(self, applied, shortcut)
1113 """Recombine the applied objects like the original."""
1114 applied_example, applied = peek_at(applied)
-> 1115 coord, dim, positions = self._infer_concat_args(applied_example)
1116 if shortcut:
1117 combined = self._concat_shortcut(applied, dim, positions)
File ~/Work/miniconda3/envs/20211123_py39/lib/python3.9/site-packages/xarray/core/groupby.py:559, in GroupBy._infer_concat_args(self, applied_example)
558 def _infer_concat_args(self, applied_example):
--> 559 if self._group_dim in applied_example.dims:
560 coord = self._group
561 positions = self._group_indices
AttributeError: 'numpy.float64' object has no attribute 'dims'
Anything else we need to know?
No response
Environment
INSTALLED VERSIONS
commit: None
python: 3.9.13 | packaged by conda-forge | (main, May 27 2022, 16:56:21)
[GCC 10.3.0]
python-bits: 64
OS: Linux
OS-release: 4.4.0-210-generic
machine: x86_64
processor: x86_64
byteorder: little
LC_ALL: None
LANG: C
LOCALE: ('en_GB', 'UTF-8')
libhdf5: 1.12.2
libnetcdf: 4.8.1
xarray: 2022.6.0
pandas: 1.4.3
numpy: 1.23.2
scipy: 1.9.0
netCDF4: 1.6.0
pydap: installed
h5netcdf: None
h5py: 3.7.0
Nio: None
zarr: 2.12.0
cftime: 1.6.1
nc_time_axis: None
PseudoNetCDF: None
rasterio: None
cfgrib: None
iris: None
bottleneck: None
dask: 2022.6.1
distributed: 2022.6.1
matplotlib: 3.5.3
cartopy: 0.20.3
seaborn: None
numbagg: None
fsspec: 2022.7.1
cupy: None
pint: None
sparse: None
flox: None
numpy_groupies: None
setuptools: 65.2.0
pip: 22.2.2
conda: 4.14.0
pytest: 7.1.2
IPython: 8.4.0
sphinx: 5.1.1
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Reproduce the minimal example with DataArrayResample.apply() and inspect the apply/map entry points in xarray/core/resample.py, followed into the combination logic in xarray/core/groupby.py. Confirm that applying np.median to resampled data returns the expected resampled DataArray rather than the reported AttributeError, and add regression coverage for this behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- numpy, pandas, python
- Domain
- data
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Clearly specified
- Newbie friendliness
- 45/100