coordinates not removed for variable encoding during reset_coords
Open
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 4.2k
- Forks
- 1.4k
- Avg merge
- 2d 15h
- Merged PRs (30d)
- 14
Description
What happened?
When calling reset_coords on a dataset that is loaded from disk, the coordinates are not removed from the encoding of the variable.
This means, that at save time they will be resaved as coordinates... annoying. (and erroneous)
What did you expect to happen?
No response
Minimal Complete Verifiable Example
import xarray as xr
dataset = xr.Dataset(
data_vars={'images': (('y', 'x'), np.zeros((10, 2)))},
coords={'zar': 1}
)
dataset.to_netcdf('foo.nc', mode='w')
# %%
foo_loaded = xr.open_dataset('foo.nc')
foo_loaded_reset = foo_loaded.reset_coords()
# %%
assert 'zar' in foo_loaded.coords
assert 'zar' not in foo_loaded_reset.coords
assert 'zar' in foo_loaded_reset.data_vars
foo_loaded_reset.to_netcdf('bar.nc', mode='w')
# %% Now load the dataset
bar_loaded = xr.open_dataset('bar.nc')
assert 'zar' not in bar_loaded.coords, 'zar is erroneously a coordinate'
# %%
# This is the problem
assert 'zar' not in foo_loaded_reset.images.encoding['coordinates'].split(' '), "zar should not be in here"
MVCE confirmation
- Minimal example — the example is as focused as reasonably possible to demonstrate the underlying issue in xarray.
- Complete example — the example is self-contained, including all data and the text of any traceback.
- Verifiable example — the example copy & pastes into an IPython prompt or Binder notebook, returning the result.
- New issue — a search of GitHub Issues suggests this is not a duplicate.
Relevant log output
No response
Anything else we need to know?
for _, variable in obj._variables.items():
coords_in_encoding = set(variable.encoding.get('coordinates', ' ').split(' '))
variable.encoding['coordinates'] = ' '.join(coords_in_encoding - set(names))
suggested fix in dataset.py, reset_coords
Environment
INSTALLED VERSIONS
------------------
commit: None
python: 3.9.13 | packaged by Ramona Optics | (main, Aug 31 2022, 22:30:30)
[GCC 10.4.0]
python-bits: 64
OS: Linux
OS-release: 5.15.0-50-generic
machine: x86_64
processor: x86_64
byteorder: little
LC_ALL: None
LANG: en_US.UTF-8
LOCALE: ('en_US', 'UTF-8')
libhdf5: 1.12.2
libnetcdf: 4.8.1
xarray: 2022.10.0
pandas: 1.5.1
numpy: 1.23.4
scipy: 1.9.3
netCDF4: 1.6.1
pydap: None
h5netcdf: 1.0.2
h5py: 3.7.0
Nio: None
zarr: 2.13.3
cftime: 1.6.2
nc_time_axis: None
PseudoNetCDF: None
rasterio: None
cfgrib: None
iris: None
bottleneck: None
dask: 2022.10.0
distributed: 2022.10.0
matplotlib: 3.6.1
cartopy: None
seaborn: None
numbagg: None
fsspec: 2022.10.0
cupy: None
pint: 0.20.1
sparse: None
flox: None
numpy_groupies: None
setuptools: 65.5.0
pip: 22.3
conda: 22.9.0
pytest: 7.2.0
IPython: 7.33.0
sphinx: 5.3.0
/home/mark/mambaforge/envs/mcam_dev/lib/python3.9/site-packages/_distutils_hack/__init__.py:33: UserWarning: Setuptools is replacing distutils.
warnings.warn("Setuptools is replacing distutils.")
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in xarray/core/dataset.py at reset_coords, then reproduce the provided netCDF round-trip example. Check the variable encoding after coordinates are reset; done means the removed coordinate is absent from the encoding and is not restored as a coordinate when the result is saved and reopened.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- data
- Issue type
- Bug
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Stale
- Clarity
- Clearly specified
- Newbie friendliness
- 55/100