Bottleneck and dask objects ignore `min_periods` on `rolling`
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 4.2k
- Forks
- 1.4k
- Avg merge
- 2d 15h
- Merged PRs (30d)
- 14
Description
What happened:
When bottleneck is installed in an environment, it seems to ignore the min_periods kwarg on ds.rolling(...).
What you expected to happen:
When using ds.rolling(..., min_periods=1), it should be able to handle an array of length 1. Without bottleneck installed, it returns the original value of a length 1 array. With bottleneck installed, the error is:
ValueError: Moving window (=2) must between 1 and 1, inclusive
Minimal Complete Verifiable Example:
With bottleneck installed to environment:
import xarray as xr
ds = xr.DataArray([1], dims='time')
ds.rolling(time=2, center=True, min_periods=1).mean()
ValueError: Moving window (=2) must between 1 and 1, inclusive
Without bottleneck installed to environment:
import xarray as xr
ds = xr.DataArray([1], dims='time')
ds.rolling(time=2, center=True, min_periods=1).mean()
>>> <xarray.DataArray (time: 1)>
>>> array([1.])
>>> Dimensions without coordinates: time
Anything else we need to know?:
In an applied case, this came up while working on .groupby('time.dayofyear').map(_rolling), where we map a rolling mean function over a defined N days with min_periods=1. Some climatological days (like leap years) will not have the N day requirement, so the min_period catch handles that, but with bottleneck installed it breaks due to the above issue.
Environment:
Output of xr.show_versions()
INSTALLED VERSIONS ------------------ commit: None python: 3.8.6 | packaged by conda-forge | (default, Jan 25 2021, 23:22:12) [Clang 11.0.1 ] python-bits: 64 OS: Darwin OS-release: 19.6.0 machine: x86_64 processor: i386 byteorder: little LC_ALL: en_US.UTF-8 LANG: en_US.UTF-8 LOCALE: en_US.UTF-8 libhdf5: 1.10.6 libnetcdf: 4.7.4xarray: 0.16.2
pandas: 1.2.1
numpy: 1.19.5
scipy: 1.6.0
netCDF4: 1.5.5.1
pydap: None
h5netcdf: 0.8.1
h5py: 3.1.0
Nio: None
zarr: 2.6.1
cftime: 1.3.1
nc_time_axis: 1.2.0
PseudoNetCDF: None
rasterio: 1.1.8
cfgrib: None
iris: None
bottleneck: 1.3.2
dask: 2021.01.1
distributed: 2021.01.1
matplotlib: 3.3.3
cartopy: 0.18.0
seaborn: 0.11.1
numbagg: None
pint: 0.16.1
setuptools: 49.6.0.post20210108
pip: 21.0
conda: None
pytest: 6.2.2
IPython: 7.18.1
sphinx: 3.4.3
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by running the minimal xarray example with and without bottleneck, then trace the rolling implementation and its bottleneck execution path. Compare both results for a length-one array with window 2 and min_periods=1; done means the bottleneck path matches the non-bottleneck result without raising the window-size error.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- data
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 38/100