pydata / pydata/xarray

`.sel()` fails on `datetime64[s]` object

Open
#10,975 5 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug upstream issue
Dominant language
Python
Stars
4.2k
Forks
1.4k
Avg merge
2d 15h
Merged PRs (30d)
14

Description

What happened?

Hi there,

sorry if this might be a duplicate, but I have been browsing the repo without finding anything specific which resemble this.

So, I am exploring to the possibility of calling xarray with CFDatetimeDecoder on time period overshooting pandas threshold year 2262
Running with xarray=2025.9.0

import xarray as xr
files="/ec/res4/scratch/ecme3497/ece4/pic2/output/oifs/pic2_atm_cmip6_1m_24*.nc"
coder = xr.coders.CFDatetimeCoder(time_unit='s')
data = xr.open_mfdataset(files, decode_times=coder)
data.time_counter

which gives me

array(['2400-01-16T12:00:00', '2400-02-15T12:00:00', '2400-03-16T12:00:00',
       ..., '2489-10-16T12:00:00', '2489-11-16T00:00:00',
       '2489-12-16T12:00:00'], shape=(1080,), dtype='datetime64[s]')

so far so good. However

data.sel(time_counter=slice("2400-01-01", "2420-01-01"))
OverflowError                             Traceback (most recent call last)
File pandas/_libs/tslibs/period.pyx:1169, in pandas._libs.tslibs.period.period_ordinal_to_dt64()

OverflowError: Overflow occurred in npy_datetimestruct_to_datetime

The above exception was the direct cause of the following exception:

OutOfBoundsDatetime                       Traceback (most recent call last)
Cell In[8], [line 1](vscode-notebook-cell:?execution_count=8&line=1)
----> [1](vscode-notebook-cell:?execution_count=8&line=1) data.sel(time_counter=slice("2400-01-01", "2420-01-01"))

File /ECMWF_kQYjfeo/miniforge/envs/env1/lib/python3.12/site-packages/xarray/core/dataset.py:2974, in Dataset.sel(self, indexers, method, tolerance, drop, **indexers_kwargs)
   2906 """Returns a new dataset with each array indexed by tick labels
   2907 along the specified dimension(s).
   2908 
   (...)   2971 
   2972 """
   2973 indexers = either_dict_or_kwargs(indexers, indexers_kwargs, "sel")
-> [2974](https://vscode-remote+ssh-002dremote-002bhpc-002dlogin.vscode-resource.vscode-cdn.net/ECMWF_kQYjfeo/miniforge/envs/env1/lib/python3.12/site-packages/xarray/core/dataset.py:2974) query_results = map_index_queries(
   2975     self, indexers=indexers, method=method, tolerance=tolerance
   2976 )
   2978 if drop:
   2979     no_scalar_variables = {}
...
File pandas/_libs/tslibs/period.pyx:1992, in pandas._libs.tslibs.period._Period.to_timestamp()

File pandas/_libs/tslibs/period.pyx:1172, in pandas._libs.tslibs.period.period_ordinal_to_dt64()

OutOfBoundsDatetime: Out of bounds nanosecond timestamp: 2400-01-01 00:00:00

I know this can be fixed using cftime, but I though that using datetime64[s] would have fix this issue. In general, I am quite confused on what are the currently supported features of xarray for going beyond the the nanoseconds limitation of pandas. Shall we keep working with cftime? Is there something I am missing? To what extent can we used the Coder if this is the output?
Any help, also suggestion to documentation, is greatly appreciated!
Thanks a lot

EDIT: I can upload some of the data required, but below you can find a reproducible example

What did you expect to happen?

As expected, correct time selection.

Minimal Complete Verifiable Example
import numpy as np
import xarray as xr

# monthly starts from 2389-01 to 2489-12 (inclusi)
months = np.arange("2389-01", "2490-01", dtype="datetime64[M]")  # dtype M = month starts
times = months.astype("datetime64[s]")  # convert to seconds (YYYY-MM-01T00:00:00)
# creare DataArray di esempio
da = xr.DataArray(np.zeros(times.size), coords={"time": times}, dims=["time"])
da.sel(time=slice("2400-01-01", "2420-01-01"))

I have been able to replicate this also in most recent version of xarray

>>> xr.show_versions()

INSTALLED VERSIONS
------------------
commit: None
python: 3.14.0 | packaged by conda-forge | (main, Dec  2 2025, 20:23:19) [Clang 20.1.8 ]
python-bits: 64
OS: Darwin
OS-release: 23.6.0
machine: arm64
processor: arm
byteorder: little
LC_ALL: None
LANG: it_IT.UTF-8
LOCALE: ('it_IT', 'UTF-8')
libhdf5: None
libnetcdf: None

xarray: 2025.11.0
pandas: 2.3.3
numpy: 2.3.5
scipy: None
netCDF4: None
pydap: None
h5netcdf: None
h5py: None
zarr: None
cftime: None
nc_time_axis: None
iris: None
bottleneck: None
dask: None
distributed: None
matplotlib: None
cartopy: None
seaborn: None
numbagg: None
fsspec: None
cupy: None
pint: None
sparse: None
flox: None
numpy_groupies: None
setuptools: None
pip: 25.3
conda: None
pytest: None
mypy: None
IPython: None
sphinx: None
MVCE confirmation
  • Minimal example — the example is as focused as reasonably possible to demonstrate the underlying issue in xarray.
  • Complete example — the example is self-contained, including all data and the text of any traceback.
  • Verifiable example — the example copy & pastes into an IPython prompt or Binder notebook, returning the result.
  • New issue — a search of GitHub Issues suggests this is not a duplicate.
  • Recent environment — the issue occurs with the latest version of xarray and its dependencies.
Environment

xarray>=2025.9.0

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the provided NumPy/xarray MVCE and trace the time-coordinate path used by DataArray.sel for string slices and datetime64[s] values. Confirm the failure against the shown pandas traceback; done means the 2400–2420 selection succeeds without requiring cftime and is covered by a regression test.

Written by the indexing model from the issue text.

Assessment

Tech stack
numpy, pandas, python
Domain
data
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.