DataArrayCoarsen.reduce passes kwargs to coordinate function
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 4.2k
- Forks
- 1.4k
- Avg merge
- 2d 15h
- Merged PRs (30d)
- 14
Description
What is your issue?
When using custom reducing function with the .reduce(func, keep_attrs=None, **kwargs) method of xarray.core.rolling.DataArrayCoarsen object, keyword arguments of the reduce method are passed to the given reducing function (func) as documented. However, if a different function is given to reduce the coordinates when the coarsen or rolling object is created, e.g. coord_func="mean", the same kwargs get also passed to the coordinate reducing function. This is quite often undesired, as the function to reduce the coordinates might not have the same kwargs as the function used to reduce the actual data.
Here is an example:
import numpy as np
import xarray as xr
def reduce_func_with_kwargs(x, axis, dummy=None):
dummy = 0
return np.sum(x, axis=axis)
np.random.seed(0)
temperature = 15 + 8 * np.random.randn(2, 2)
lon = [[-99.83, -99.32], [-99.79, -99.23]]
lat = [[42.25, 42.21], [42.63, 42.59]]
da = xr.DataArray(
data=temperature,
dims=["x", "y"],
coords=dict(
lon=(["x", "y"], lon),
lat=(["x", "y"], lat),
),
attrs=dict(
description="Ambient temperature.",
units="degC",
),
)
# This works fine
coarsen_mean = da.coarsen({"x": 2, "y": 2}, coord_func="mean").reduce(
reduce_func_with_kwargs
)
# This raises TypeError: nanmean() got an unexpected keyword argument 'dummy'
coarsen_reduce = da.coarsen({"x": 2, "y": 2}, coord_func="mean").reduce(
reduce_func_with_kwargs, dummy=1
)
The first reduction works as expected, as no kwargs are passed to the reduce method. However, the latter reduction raises a TypeError
Traceback (most recent call last):
File "/home/saskartt/Code/misc/xarray_bug_example.py", line 28, in <module>
coarsen_reduce = da.coarsen({"x": 2, "y": 2}, coord_func="mean").reduce(reduce_func_with_kwargs, dummy=1)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/saskartt/miniconda3/envs/xarray-bug/lib/python3.11/site-packages/xarray/core/rolling.py", line 1064, in reduce
return wrapped_func(self, keep_attrs=keep_attrs, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/saskartt/miniconda3/envs/xarray-bug/lib/python3.11/site-packages/xarray/core/rolling.py", line 1012, in wrapped_func
coords[c] = v.variable.coarsen(
^^^^^^^^^^^^^^^^^^^
File "/home/saskartt/miniconda3/envs/xarray-bug/lib/python3.11/site-packages/xarray/core/variable.py", line 2522, in coarsen
return self._replace(data=func(reshaped, axis=axes, **kwargs), attrs=_attrs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/saskartt/miniconda3/envs/xarray-bug/lib/python3.11/site-packages/xarray/core/duck_array_ops.py", line 615, in mean
return _mean(array, axis=axis, skipna=skipna, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/saskartt/miniconda3/envs/xarray-bug/lib/python3.11/site-packages/xarray/core/duck_array_ops.py", line 382, in f
return func(values, axis=axis, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: nanmean() got an unexpected keyword argument 'dummy'
This is because kwarg dummy=1 gets passed to coordinate reduction function "mean" (nanmean) in addition to reduce_func_with_kwargs in xarray/core/rolling.py:1012.
Currently, it is undocumented that the kwargs are also passed to the coordinate reduction function, so this is probably technically not a bug. However, I think in many cases you might want to use different kwargs for the functions and some solution for this case would be nice.
INSTALLED VERSIONS
------------------
commit: None
python: 3.11.4 | packaged by conda-forge | (main, Jun 10 2023, 18:08:17) [GCC 12.2.0]
xarray: 2023.7.0
pandas: 2.0.3
numpy: 1.25.2
Cheers,
Sasu Karttunen
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in xarray/core/rolling.py around line 1012 and reproduce the DataArrayCoarsen.reduce example with coord_func="mean" and dummy=1. Trace how kwargs reach the data and coordinate reduction functions; done when the custom reducer still receives its kwargs without causing the coordinate reducer to raise the reported TypeError.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- numpy, python
- Domain
- data
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100