Short-term archiver mis-identifies latest set of restarts when crossing y10k boundary
- Dominant language
- Python
- Stars
- 174
- Forks
- 225
- Avg merge
- 1d 16h
- Merged PRs (30d)
- 14
Description
(This bug was found with cime_cesm2_0_rel_04, but it looks to me like the same bug would exist on cime master.)
When running a case that crossed the y10k boundary - i.e., going from year 9000-something to year 10000-something, @d80ngv hit an issue with the short-term archiver: It seems to have mis-identified the latest set of restart files, so it threw away the restarts with year 10000-something, and re-copied the restarts with year 9000-something. Here is the relevant output from the starchive script, for a run that went from year 9078 to year 10078:
```
-------------------------------------------
Archiving restarts for date date(10078, 1, 1, 0, 0, 0)
-------------------------------------------
Archiving restarts for dlnd (lnd)
Archiving restarts for cism (glc)
removing interim restart file /glade/scratch/sbradley//t.virtEC_noshelves_wd150_phi10_noicecaps/run/t.virtEC_noshelves_wd150_phi10_noicecaps.cism.r.10078-01-01-00000.nc
Archiving restarts for drv (cpl)
removing interim restart file /glade/scratch/sbradley//t.virtEC_noshelves_wd150_phi10_noicecaps/run/t.virtEC_noshelves_wd150_phi10_noicecaps.cpl.r.10078-01-01-00000.nc
Archiving restarts for dart (esp)
-------------------------------------------
Archiving restarts for date date(9078, 1, 1, 0, 0, 0)
-------------------------------------------
Archiving restarts for dlnd (lnd)
Archiving restarts for cism (glc)
copying file /glade/scratch/sbradley//t.virtEC_noshelves_wd150_phi10_noicecaps/run/t.virtEC_noshelves_wd150_phi10_noicecaps.cism.r.9078-01-01-00000.nc to /glade/scratch/sbradley//archive/t.virtEC_noshelves_wd150_phi10_noicecaps/rest/9078-01-01-00000/t.virtEC_noshelves_wd150_phi10_noicecaps.cism.r.9078-01-01-00000.nc
Archiving restarts for drv (cpl)
copying file /glade/scratch/sbradley//t.virtEC_noshelves_wd150_phi10_noicecaps/run/t.virtEC_noshelves_wd150_phi10_noicecaps.cpl.r.9078-01-01-00000.nc to /glade/scratch/sbradley//archive/t.virtEC_noshelves_wd150_phi10_noicecaps/rest/9078-01-01-00000/t.virtEC_noshelves_wd150_phi10_noicecaps.cpl.r.9078-01-01-00000.nc
```
Looking through the `case_st_archive` code, my best guess of what's happening is: `_get_datenames` returns an alphabetically-sorted list of cpl restart file names, then the scripts treat all but the last as being "interim" restart files. However, this results in a file with year 10078 coming before a file with year 9078, because we don't use leading 0s for 4-digit years.
A possible solution would be to sort the datenames returned by `_get_datenames` - i.e., replace `return datenames` with `return sorted(datenames)`. However, I haven't looked closely to determine if:
1. This would have any unintended consequences
2. This is sufficient to fix the problem
Contributor guide
Assessment
This issue has not been assessed yet.