Dataset.weighted along a dimension not on weights errors
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 4.2k
- Forks
- 1.4k
- Avg merge
- 2d 15h
- Merged PRs (30d)
- 14
Description
What happened?
ds.weighted(weights).mean(dims) errors when reducing over a dimension that is neither on the weights nor on the variable.
What did you expect to happen?
This used to work and was "broken" by #8606. However, we may want to fix this by ignoring (?) those data vars instead (#7027).
Minimal Complete Verifiable Example
import xarray as xr
ds = xr.Dataset({"a": (("y", "x"), [[1, 2]]), "scalar": 1})
weights = xr.DataArray([1, 2], dims="x")
ds.weighted(weights).mean("y")
MVCE confirmation
- Minimal example — the example is as focused as reasonably possible to demonstrate the underlying issue in xarray.
- Complete example — the example is self-contained, including all data and the text of any traceback.
- Verifiable example — the example copy & pastes into an IPython prompt or Binder notebook, returning the result.
- New issue — a search of GitHub Issues suggests this is not a duplicate.
- Recent environment — the issue occurs with the latest version of xarray and its dependencies.
Relevant log output
ValueError Traceback (most recent call last)
Cell In[1], line 6
3 ds = xr.Dataset({"a": (("y", "x"), [[1, 2]]), "scalar": 1})
4 weights = xr.DataArray([1, 2], dims="x")
----> 6 ds.weighted(weights).mean("y")
File ~/code/xarray/xarray/util/deprecation_helpers.py:115, in _deprecate_positional_args.<locals>._decorator.<locals>.inner(*args, **kwargs)
111 kwargs.update({name: arg for name, arg in zip_args})
113 return func(*args[:-n_extra_args], **kwargs)
--> 115 return func(*args, **kwargs)
File ~/code/xarray/xarray/core/weighted.py:497, in Weighted.mean(self, dim, skipna, keep_attrs)
489 @_deprecate_positional_args("v2023.10.0")
490 def mean(
491 self,
(...)
495 keep_attrs: bool | None = None,
496 ) -> T_Xarray:
--> 497 return self._implementation(
498 self._weighted_mean, dim=dim, skipna=skipna, keep_attrs=keep_attrs
499 )
File ~/code/xarray/xarray/core/weighted.py:558, in DatasetWeighted._implementation(self, func, dim, **kwargs)
555 def _implementation(self, func, dim, **kwargs) -> Dataset:
556 self._check_dim(dim)
--> 558 return self.obj.map(func, dim=dim, **kwargs)
File ~/code/xarray/xarray/core/dataset.py:6924, in Dataset.map(self, func, keep_attrs, args, **kwargs)
6922 if keep_attrs is None:
6923 keep_attrs = _get_keep_attrs(default=False)
-> 6924 variables = {
6925 k: maybe_wrap_array(v, func(v, *args, **kwargs))
6926 for k, v in self.data_vars.items()
6927 }
6928 if keep_attrs:
6929 for k, v in variables.items():
File ~/code/xarray/xarray/core/dataset.py:6925, in <dictcomp>(.0)
6922 if keep_attrs is None:
6923 keep_attrs = _get_keep_attrs(default=False)
6924 variables = {
-> 6925 k: maybe_wrap_array(v, func(v, *args, **kwargs))
6926 for k, v in self.data_vars.items()
6927 }
6928 if keep_attrs:
6929 for k, v in variables.items():
File ~/code/xarray/xarray/core/weighted.py:286, in Weighted._weighted_mean(self, da, dim, skipna)
278 def _weighted_mean(
279 self,
280 da: T_DataArray,
281 dim: Dims = None,
282 skipna: bool | None = None,
283 ) -> T_DataArray:
284 """Reduce a DataArray by a weighted ``mean`` along some dimension(s)."""
--> 286 weighted_sum = self._weighted_sum(da, dim=dim, skipna=skipna)
288 sum_of_weights = self._sum_of_weights(da, dim=dim)
290 return weighted_sum / sum_of_weights
File ~/code/xarray/xarray/core/weighted.py:276, in Weighted._weighted_sum(self, da, dim, skipna)
268 def _weighted_sum(
269 self,
270 da: T_DataArray,
271 dim: Dims = None,
272 skipna: bool | None = None,
273 ) -> T_DataArray:
274 """Reduce a DataArray by a weighted ``sum`` along some dimension(s)."""
--> 276 return self._reduce(da, self.weights, dim=dim, skipna=skipna)
File ~/code/xarray/xarray/core/weighted.py:231, in Weighted._reduce(da, weights, dim, skipna)
227 da = da.fillna(0.0)
229 # `dot` does not broadcast arrays, so this avoids creating a large
230 # DataArray (if `weights` has additional dimensions)
--> 231 return dot(da, weights, dim=dim)
File ~/code/xarray/xarray/util/deprecation_helpers.py:140, in deprecate_dims.<locals>.wrapper(*args, **kwargs)
132 emit_user_level_warning(
133 "The `dims` argument has been renamed to `dim`, and will be removed "
134 "in the future. This renaming is taking place throughout xarray over the "
(...)
137 PendingDeprecationWarning,
138 )
139 kwargs["dim"] = kwargs.pop("dims")
--> 140 return func(*args, **kwargs)
File ~/code/xarray/xarray/core/computation.py:1885, in dot(dim, *arrays, **kwargs)
1883 dim = tuple(d for d, c in dim_counts.items() if c > 1)
1884 else:
-> 1885 dim = parse_dims(dim, all_dims=tuple(all_dims))
1887 dot_dims: set[Hashable] = set(dim)
1889 # dimensions to be parallelized
File ~/code/xarray/xarray/core/utils.py:1046, in parse_dims(dim, all_dims, check_exists, replace_none)
1044 dim = (dim,)
1045 if check_exists:
-> 1046 _check_dims(set(dim), set(all_dims))
1047 return tuple(dim)
File ~/code/xarray/xarray/core/utils.py:1131, in _check_dims(dim, all_dims)
1129 if wrong_dims:
1130 wrong_dims_str = ", ".join(f"'{d!s}'" for d in wrong_dims)
-> 1131 raise ValueError(
1132 f"Dimension(s) {wrong_dims_str} do not exist. Expected one or more of {all_dims}"
1133 )
ValueError: Dimension(s) 'y' do not exist. Expected one or more of {'x'}
Anything else we need to know?
No response
Environment
Newest main (i.e. 2024.01)
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Reproduce the MVCE, then read xarray/core/weighted.py around DatasetWeighted._implementation, Weighted._reduce, and the call into Dataset.map and dot. Determine the intended handling for variables missing the reduced dimension, add regression coverage for ds.weighted(weights).mean("y"), and verify the result matches the chosen behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- data
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 42/100