weighted polyfit with NaNs appears broken
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 4.2k
- Forks
- 1.4k
- Avg merge
- 2d 15h
- Merged PRs (30d)
- 14
Description
What happened?
polyfit can't seem to handle weights still (despite https://github.com/pydata/xarray/pull/5933). nans are involved.
What did you expect to happen?
No response
Minimal Complete Verifiable Example
np.random.seed(10)
y = np.random.rand(10)
x = np.arange(len(y))
y[3] = np.nan
y[5] = np.nan
w = y/10
msk = ~np.isnan(y)
x = x[msk]
y = y[msk]
w = w[msk]
da_test = xr.DataArray(y, dims='time', coords={'time': x})
da_test.polyfit('time', 1, w=w)
# np.polyfit(x, y, 1)#, w=w) # <-- should match this
MVCE confirmation
- Minimal example — the example is as focused as reasonably possible to demonstrate the underlying issue in xarray.
- Complete example — the example is self-contained, including all data and the text of any traceback.
- Verifiable example — the example copy & pastes into an IPython prompt or Binder notebook, returning the result.
- New issue — a search of GitHub Issues suggests this is not a duplicate.
- Recent environment — the issue occurs with the latest version of xarray and its dependencies.
Relevant log output
No response
Anything else we need to know?
not inserting the nans and/or converting the weights to a data array didn't matter
Environment
INSTALLED VERSIONS
commit: None
python: 3.12.3 | packaged by conda-forge | (main, Apr 15 2024, 18:38:13) [GCC 12.3.0]
python-bits: 64
OS: Linux
OS-release: 5.14.0-284.30.1.el9_2.x86_64
machine: x86_64
processor: x86_64
byteorder: little
LC_ALL: None
LANG: en_US.UTF-8
LOCALE: ('en_US', 'UTF-8')
libhdf5: 1.14.3
libnetcdf: 4.9.2
xarray: 2024.3.0
pandas: 2.2.2
numpy: 1.26.4
scipy: 1.13.0
netCDF4: 1.6.5
pydap: None
h5netcdf: 1.3.0
h5py: 3.11.0
Nio: None
zarr: None
cftime: 1.6.3
nc_time_axis: None
iris: None
bottleneck: None
dask: 2024.5.0
distributed: 2024.5.0
matplotlib: 3.8.4
cartopy: 0.23.0
seaborn: 0.13.2
numbagg: None
fsspec: 2024.3.1
cupy: None
pint: None
sparse: None
flox: None
numpy_groupies: None
setuptools: 69.5.1
pip: 24.0
conda: None
pytest: None
mypy: None
IPython: 8.22.2
sphinx: None
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start at the DataArray.polyfit entry point and run the provided minimal example with the shown weighted, NaN-filtered arrays. Compare its result with numpy.polyfit using the same x, y, and w inputs, then add or update coverage so weighted fitting produces the expected matching result.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- numpy, python
- Domain
- data
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100