NCAS-CMS / NCAS-CMS/cf-python

reading pp files with non-gregorian calendars does not work

Open
#170 4 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug um/pp
Dominant language
Python
Stars
150
Forks
23
Avg merge
1d 11h
Merged PRs (30d)
2

Description

I am attempting to aggregate as set of UM pp files in a directory.
These include a file which appears to have a 30th of February.
I had expected cf-python to warn me and change to a 360_day calendar
(or allow me to specify it in the read statement).

Error message

 File "/home/users/lawrence/miniconda3/envs/macbook/lib/python3.7/site-packages/cf/read_write/um/umread.py", line 678, in __init__
    c = self.time_coordinate(axiscode)
  File "/home/users/lawrence/miniconda3/envs/macbook/lib/python3.7/site-packages/cf/read_write/um/umread.py", line 2201, in time_coordinate
    vtimes = numpy_array([self.vtime(rec) for rec in recs], dtype=float)
  File "/home/users/lawrence/miniconda3/envs/macbook/lib/python3.7/site-packages/cf/read_write/um/umread.py", line 2201, in <listcomp>
    vtimes = numpy_array([self.vtime(rec) for rec in recs], dtype=float)
  File "/home/users/lawrence/miniconda3/envs/macbook/lib/python3.7/site-packages/cf/read_write/um/umread.py", line 2320, in vtime
    time = netCDF4_date2num(cftime.datetime(*LBVTIME),
 "cftime/_cftime.pyx", line 948, in cftime._cftime.datetime.__init__
  File "cftime/_cftime.pyx", line 1492, in cftime._cftime.assert_valid_date
ValueError: invalid day number provided in cftime.datetime(1992, 2, 30, 0, 0, 0, 0, calendar='gregorian')

Environment: sci6.jasmin.ac.uk, private conda install:

Platform: Linux-3.10.0-1127.19.1.el7.x86_64-x86_64-with-centos-7.8.2003-Core
HDF5 library: 1.10.6
netcdf library: 4.7.4
udunits2 library: /home/users/lawrence/miniconda3/envs/macbook/lib/libudunits2.so.0
python: 3.7.9
netCDF4: 1.5.3
cftime: 1.3.0
numpy: 1.19.4
psutil: 5.7.3
scipy: 1.5.3
matplotlib: 3.3.3
ESMF: not available
cfdm: 1.7.11
cfunits: 3.2.4
cfplot: 3.0.38
cf: 3.1.0

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with cf.read aggregation of UM pp files and trace the failure through cf/read_write/um/umread.py, especially time_coordinate and vtime. Reproduce the 30 February case with the reported environment or equivalent test data, then verify that the read handles the non-Gregorian calendar as requested or reports the limitation clearly.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.