New filesize limit for svn inputdata that restricts our larger ctsm5.3.0 files being rimported
- Dominant language
- Fortran
- Stars
- 352
- Forks
- 361
- Avg merge
- 2d 21h
- Merged PRs (30d)
- 7
Description
Currently the svn server in CGD for inputdata has a file size limit for importing new files that is around 23GBytes. When we rimported ctsm5.2.0 files we put in files up to 114GB and did NOT see this problem.
This came up in ctsm5.3.0 work in #2500 as referenced below.
In our CSEG meeting we decided we need to transition to a new inputdata storage solution anyway. So we plan to leave these files OUT of svn inputdata until later.
@ekluzek I have started the last ./rimport but these landuse files returned an error, which makes me think that all that f09 landuse files (and possibly others) are too big:
```
svn: E175002: Unexpected HTTP status 413 'Request Entity Too Large' on '/!svn/txr/68458-1k6x/trunk/inputdata/lnd/clm2/surfdata_esmf/ctsm5.3.0/landuse.timeseries_0.9x1.25_hist_1700-2023_78pfts_c240908.nc'
svn: E175002: Unexpected HTTP status 413 'Request Entity Too Large' on '/!svn/txr/68458-1k6y/trunk/inputdata/lnd/clm2/surfdata_esmf/ctsm5.3.0/landuse.timeseries_0.9x1.25_SSP1-1.9_1850-2100_78pfts_c240908.nc'
svn: E175002: Unexpected HTTP status 413 'Request Entity Too Large' on '/!svn/txr/68458-1k6z/trunk/inputdata/lnd/clm2/surfdata_esmf/ctsm5.3.0/landuse.timeseries_0.9x1.25_SSP1-2.6_1850-2100_78pfts_c240908.nc'
```
UPDATE: Some give a more vague error such as this:
`svn: E175002: PUT request on '/!svn/txr/68459-1k71/trunk/inputdata/lnd/clm2/surfdata_esmf/ctsm5.3.0/landuse.timeseries_0.9x1.25_SSP2-4.5_1850-2100_78pfts_c240908.nc' failed`
Some non-f09 files that failed with the same errors:
```
svn: E175002: PUT request on '/!svn/txr/68462-1k79/trunk/inputdata/lnd/clm2/surfdata_esmf/ctsm5.3.0/landuse.timeseries_360x720cru_SSP2-4.5_1850-2100_78pfts_c240908.nc' failed
svn: E175002: PUT request on '/!svn/txr/68463-1k7b/trunk/inputdata/lnd/clm2/surfdata_esmf/ctsm5.3.0/landuse.timeseries_C96_SSP2-4.5_1850-2100_78pfts_c240908.nc' failed
svn: E175002: Unexpected HTTP status 413 'Request Entity Too Large' on '/!svn/txr/68463-1k7c/trunk/inputdata/lnd/clm2/surfdata_esmf/ctsm5.3.0/landuse.timeseries_mpasa120_SSP2-4.5_1850-2100_78pfts_c240908.nc'
svn: E175002: PUT request on '/!svn/txr/68465-1k7f/trunk/inputdata/lnd/clm2/surfdata_esmf/ctsm5.3.0/landuse.timeseries_ne0np4CONUS.ne30x8_SSP2-4.5_1979-2026_78pfts_c240908.nc' failed
svn: E175002: Unexpected HTTP status 413 'Request Entity Too Large' on '/!svn/txr/68468-1k7j/trunk/inputdata/lnd/clm2/surfdata_esmf/ctsm5.3.0/landuse.timeseries_ne30np4.pg3_SSP2-4.5_1850-2100_78pfts_c240908.nc'
svn: E175002: Unexpected HTTP status 413 'Request Entity Too Large' on '/!svn/txr/68468-1k7k/trunk/inputdata/lnd/clm2/surfdata_esmf/ctsm5.3.0/landuse.timeseries_ne30np4_SSP2-4.5_1850-2100_78pfts_c240908.nc'
svn: E175002: PUT request on '/!svn/txr/68496-1k8d/trunk/inputdata/lnd/clm2/surfdata_esmf/ctsm5.3.0/surfdata_mpasa15_hist_2000_16pfts_c240908.nc' failed
svn: E175002: PUT request on '/!svn/txr/68496-1k8e/trunk/inputdata/lnd/clm2/surfdata_esmf/ctsm5.3.0/surfdata_mpasa3p75_hist_2000_16pfts_c240908.nc' failed
svn: E175002: Unexpected HTTP status 413 'Request Entity Too Large' on '/!svn/txr/68507-1k8q/trunk/inputdata/lnd/clm2/surfdata_esmf/ctsm5.3.0/surfdata_ne120np4.pg3_hist_1850_78pfts_c240908.nc'
svn: E175002: PUT request on '/!svn/txr/68507-1k8r/trunk/inputdata/lnd/clm2/surfdata_esmf/ctsm5.3.0/surfdata_ne120np4.pg3_hist_2000_78pfts_c240908.nc' failed
```
_Originally posted by @slevis-lmwg in https://github.com/ESCOMP/CTSM/issues/2500#issuecomment-2339266811_
Definition of done:
- [x] Have a backup copy of these files on Izumi
- [ ] This list of files is imported into the new system
- [ ] Remove the extra copy on Izumi
Contributor guide
Assessment
This issue has not been assessed yet.