ESMCI / ESMCI/cime

Short-term archiver mis-identifies latest set of restarts when crossing y10k boundary

Open
#2,903 2 comments 0 reactions 0 assignees View on GitHub
Assigned Low Priority tp: CIMElib ty: Bug
Dominant language
Python
Stars
174
Forks
225
Avg merge
1d 16h
Merged PRs (30d)
14

Description

(This bug was found with cime_cesm2_0_rel_04, but it looks to me like the same bug would exist on cime master.)

When running a case that crossed the y10k boundary - i.e., going from year 9000-something to year 10000-something, @d80ngv hit an issue with the short-term archiver: It seems to have mis-identified the latest set of restart files, so it threw away the restarts with year 10000-something, and re-copied the restarts with year 9000-something. Here is the relevant output from the starchive script, for a run that went from year 9078 to year 10078:

```
-------------------------------------------
Archiving restarts for date date(10078, 1, 1, 0, 0, 0)
-------------------------------------------
Archiving restarts for dlnd (lnd)
Archiving restarts for cism (glc)
removing interim restart file /glade/scratch/sbradley//t.virtEC_noshelves_wd150_phi10_noicecaps/run/t.virtEC_noshelves_wd150_phi10_noicecaps.cism.r.10078-01-01-00000.nc
Archiving restarts for drv (cpl)
removing interim restart file /glade/scratch/sbradley//t.virtEC_noshelves_wd150_phi10_noicecaps/run/t.virtEC_noshelves_wd150_phi10_noicecaps.cpl.r.10078-01-01-00000.nc
Archiving restarts for dart (esp)
-------------------------------------------
Archiving restarts for date date(9078, 1, 1, 0, 0, 0)
-------------------------------------------
Archiving restarts for dlnd (lnd)
Archiving restarts for cism (glc)
copying file /glade/scratch/sbradley//t.virtEC_noshelves_wd150_phi10_noicecaps/run/t.virtEC_noshelves_wd150_phi10_noicecaps.cism.r.9078-01-01-00000.nc to /glade/scratch/sbradley//archive/t.virtEC_noshelves_wd150_phi10_noicecaps/rest/9078-01-01-00000/t.virtEC_noshelves_wd150_phi10_noicecaps.cism.r.9078-01-01-00000.nc
Archiving restarts for drv (cpl)
copying file /glade/scratch/sbradley//t.virtEC_noshelves_wd150_phi10_noicecaps/run/t.virtEC_noshelves_wd150_phi10_noicecaps.cpl.r.9078-01-01-00000.nc to /glade/scratch/sbradley//archive/t.virtEC_noshelves_wd150_phi10_noicecaps/rest/9078-01-01-00000/t.virtEC_noshelves_wd150_phi10_noicecaps.cpl.r.9078-01-01-00000.nc
```

Looking through the `case_st_archive` code, my best guess of what's happening is: `_get_datenames` returns an alphabetically-sorted list of cpl restart file names, then the scripts treat all but the last as being "interim" restart files. However, this results in a file with year 10078 coming before a file with year 9078, because we don't use leading 0s for 4-digit years.

A possible solution would be to sort the datenames returned by `_get_datenames` - i.e., replace `return datenames` with `return sorted(datenames)`. However, I haven't looked closely to determine if:

1. This would have any unintended consequences

2. This is sufficient to fix the problem

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.