pydata / pydata/xarray

When decode_times fails, warn rather than failing

Open
#2,848 30 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
4.2k
Forks
1.4k
Avg merge
2d 15h
Merged PRs (30d)
14

Description

There are various reasons why time decoding can fail on a netcdf or opendap file, but the immediate solution is always to try again with decode_times=False. Therefore it would save time and frustration if instead of failing, open_dataset would simply issue a warning for any variable it could not decode, and proceed to decode those that it can. As an example, here is a URL for which the time coordinate can be converted but there is another time-related array, tau that cannot--and that probably shouldn't be, in any case:

import xarray as xr
url = "http://tds.hycom.org/thredds/dodsC/GLBu0.08/expt_90.9/uv3z"
d0 = xr.open_dataset(url, decode_times=False)
print(d0.time)
print(d0.tau)

yielding

<xarray.DataArray 'time' (time: 438)>
array([105768., 105792., 105816., ..., 119472., 119496., 119520.])
Coordinates:
  * time     (time) float64 1.058e+05 1.058e+05 ... 1.195e+05 1.195e+05
Attributes:
    long_name:    Valid Time
    units:        hours since 2000-01-01 00:00:00
    time_origin:  2000-01-01 00:00:00
    calendar:     gregorian
    axis:         T
    NAVO_code:    13

<xarray.DataArray 'tau' (time: 438)>
array([0., 0., 0., ..., 0., 0., 0.])
Coordinates:
  * time     (time) float64 1.058e+05 1.058e+05 ... 1.195e+05 1.195e+05
Attributes:
    long_name:    Tau
    units:        hours since analysis
    time_origin:  2013-08-19 00:00:00
    NAVO_code:    56
Output of xr.show_versions()

INSTALLED VERSIONS

commit: None
python: 3.6.7 | packaged by conda-forge | (default, Feb 28 2019, 02:16:08)
[GCC 4.2.1 Compatible Clang 4.0.1 (tags/RELEASE_401/final)]
python-bits: 64
OS: Darwin
OS-release: 16.7.0
machine: x86_64
processor: i386
byteorder: little
LC_ALL: None
LANG: en_US.UTF-8
LOCALE: en_US.UTF-8
libhdf5: 1.10.2
libnetcdf: 4.6.1

xarray: 0.12.0
pandas: 0.24.2
numpy: 1.16.2
scipy: 1.2.1
netCDF4: 1.4.1
pydap: None
h5netcdf: 0.6.1
h5py: 2.8.0
Nio: None
zarr: None
cftime: 1.0.0
nc_time_axis: None
PseudonetCDF: None
rasterio: None
cfgrib: None
iris: None
bottleneck: 1.2.1
dask: 0.18.2
distributed: 1.22.0
matplotlib: 2.2.2
cartopy: 0.16.0
seaborn: None
setuptools: 40.0.0
pip: 18.1
conda: 4.5.11
pytest: 3.7.1
IPython: 6.5.0
sphinx: 1.8.0

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start at open_dataset and follow the decode_times path, reproducing the provided HYCOM URL and comparing it with decode_times=False. The change is done when an undecodable variable such as tau emits a warning, other time variables still decode, and opening the dataset no longer fails solely because one variable cannot be decoded.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.