pydata / pydata/xarray

DataArrayRolling.mean() ignores `skipna=True` kwarg

Open
#6,772 6 comments 2 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug topic-rolling
Dominant language
Python
Stars
4.2k
Forks
1.4k
Avg merge
2d 15h
Merged PRs (30d)
14

Description

What happened?

In the following example, it appears that the skipna=True argument is being ignored, unless you call construct before taking the mean.

What did you expect to happen?

I expected:

da.rolling(time=3).mean("time", skipna=True) == da.rolling(time=3).construct("window").mean("window", skipna=True)
Minimal Complete Verifiable Example
import xarray as xr
import numpy as np

da = xr.DataArray([0, 1, 2, np.NAN, 4, 5, 6], dims="time")

# Results 1 and 2 produce the same array, without skipping NANs
result1 = da.rolling(time=3).mean("time", skipna=True)
result2 = da.rolling(time=3).mean("time", skipna=False)

# Results 3 and 4 produce the expected (different) arrays
result3 = da.rolling(time=3).construct("window").mean("window", skipna=True)
result4 = da.rolling(time=3).construct("window").mean("window", skipna=False)

print(result1)
print(result2)
print(result3)
print(result4)
MVCE confirmation
  • Minimal example — the example is as focused as reasonably possible to demonstrate the underlying issue in xarray.
  • Complete example — the example is self-contained, including all data and the text of any traceback.
  • Verifiable example — the example copy & pastes into an IPython prompt or Binder notebook, returning the result.
  • New issue — a search of GitHub Issues suggests this is not a duplicate.
Relevant log output
<xarray.DataArray (time: 7)>
array([nan, nan,  1., nan, nan, nan,  5.])
Dimensions without coordinates: time
<xarray.DataArray (time: 7)>
array([nan, nan,  1., nan, nan, nan,  5.])
Dimensions without coordinates: time
<xarray.DataArray (time: 7)>
array([0. , 0.5, 1. , 1.5, 3. , 4.5, 5. ])
Dimensions without coordinates: time
<xarray.DataArray (time: 7)>
array([nan, nan,  1., nan, nan, nan,  5.])
Dimensions without coordinates: time
Anything else we need to know?

Note: the first two NaNs are expected due to the min_periods argument to rolling(), but the middle NaNs are what are unexpected when skipna = True

Environment

INSTALLED VERSIONS

commit: None
python: 3.9.13 | packaged by conda-forge | (main, May 27 2022, 16:50:36) [MSC v.1929 64 bit (AMD64)]
python-bits: 64
OS: Windows
OS-release: 10
machine: AMD64
processor: AMD64 Family 23 Model 17 Stepping 0, AuthenticAMD
byteorder: little
LC_ALL: None
LANG: None
LOCALE: ('English_United States', '1252')
libhdf5: 1.12.1
libnetcdf: 4.8.1

xarray: 0.19.0
pandas: 1.3.4
numpy: 1.21.2
scipy: 1.7.1
netCDF4: 1.6.0
pydap: None
h5netcdf: None
h5py: None
Nio: None
zarr: 2.12.0
cftime: 1.6.0
nc_time_axis: None
PseudoNetCDF: None
rasterio: 1.2.10
cfgrib: None
iris: None
bottleneck: None
dask: 2022.6.1
distributed: 2022.6.1
matplotlib: 3.4.3
cartopy: 0.20.1
seaborn: None
numbagg: None
pint: 0.19.2
setuptools: 59.8.0
pip: 22.1.2
conda: None
pytest: 7.1.2
IPython: 8.4.0
sphinx: None

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by running the minimal Python example against DataArrayRolling.mean() and compare it with rolling(...).construct("window").mean(...). Check the DataArrayRolling.mean entry point and add coverage for skipna=True versus skipna=False, ensuring the rolling result matches the constructed-window result for the shown NaN input.

Written by the indexing model from the issue text.

Assessment

Tech stack
numpy, python
Domain
data
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.