ESCOMP / ESCOMP/CTSM

New filesize limit for svn inputdata that restricts our larger ctsm5.3.0 files being rimported

Open
#2,751 3 comments 0 reactions 0 assignees View on GitHub
bug enhancement
Dominant language
Fortran
Stars
352
Forks
361
Avg merge
2d 21h
Merged PRs (30d)
7

Description

Currently the svn server in CGD for inputdata has a file size limit for importing new files that is around 23GBytes. When we rimported ctsm5.2.0 files we put in files up to 114GB and did NOT see this problem.

This came up in ctsm5.3.0 work in #2500 as referenced below.

In our CSEG meeting we decided we need to transition to a new inputdata storage solution anyway. So we plan to leave these files OUT of svn inputdata until later.

@ekluzek I have started the last ./rimport but these landuse files returned an error, which makes me think that all that f09 landuse files (and possibly others) are too big:
```
svn: E175002: Unexpected HTTP status 413 'Request Entity Too Large' on '/!svn/txr/68458-1k6x/trunk/inputdata/lnd/clm2/surfdata_esmf/ctsm5.3.0/landuse.timeseries_0.9x1.25_hist_1700-2023_78pfts_c240908.nc'
svn: E175002: Unexpected HTTP status 413 'Request Entity Too Large' on '/!svn/txr/68458-1k6y/trunk/inputdata/lnd/clm2/surfdata_esmf/ctsm5.3.0/landuse.timeseries_0.9x1.25_SSP1-1.9_1850-2100_78pfts_c240908.nc'
svn: E175002: Unexpected HTTP status 413 'Request Entity Too Large' on '/!svn/txr/68458-1k6z/trunk/inputdata/lnd/clm2/surfdata_esmf/ctsm5.3.0/landuse.timeseries_0.9x1.25_SSP1-2.6_1850-2100_78pfts_c240908.nc'
```

UPDATE: Some give a more vague error such as this:
`svn: E175002: PUT request on '/!svn/txr/68459-1k71/trunk/inputdata/lnd/clm2/surfdata_esmf/ctsm5.3.0/landuse.timeseries_0.9x1.25_SSP2-4.5_1850-2100_78pfts_c240908.nc' failed`

Some non-f09 files that failed with the same errors:
```
svn: E175002: PUT request on '/!svn/txr/68462-1k79/trunk/inputdata/lnd/clm2/surfdata_esmf/ctsm5.3.0/landuse.timeseries_360x720cru_SSP2-4.5_1850-2100_78pfts_c240908.nc' failed
svn: E175002: PUT request on '/!svn/txr/68463-1k7b/trunk/inputdata/lnd/clm2/surfdata_esmf/ctsm5.3.0/landuse.timeseries_C96_SSP2-4.5_1850-2100_78pfts_c240908.nc' failed
svn: E175002: Unexpected HTTP status 413 'Request Entity Too Large' on '/!svn/txr/68463-1k7c/trunk/inputdata/lnd/clm2/surfdata_esmf/ctsm5.3.0/landuse.timeseries_mpasa120_SSP2-4.5_1850-2100_78pfts_c240908.nc'
svn: E175002: PUT request on '/!svn/txr/68465-1k7f/trunk/inputdata/lnd/clm2/surfdata_esmf/ctsm5.3.0/landuse.timeseries_ne0np4CONUS.ne30x8_SSP2-4.5_1979-2026_78pfts_c240908.nc' failed
svn: E175002: Unexpected HTTP status 413 'Request Entity Too Large' on '/!svn/txr/68468-1k7j/trunk/inputdata/lnd/clm2/surfdata_esmf/ctsm5.3.0/landuse.timeseries_ne30np4.pg3_SSP2-4.5_1850-2100_78pfts_c240908.nc'
svn: E175002: Unexpected HTTP status 413 'Request Entity Too Large' on '/!svn/txr/68468-1k7k/trunk/inputdata/lnd/clm2/surfdata_esmf/ctsm5.3.0/landuse.timeseries_ne30np4_SSP2-4.5_1850-2100_78pfts_c240908.nc'
svn: E175002: PUT request on '/!svn/txr/68496-1k8d/trunk/inputdata/lnd/clm2/surfdata_esmf/ctsm5.3.0/surfdata_mpasa15_hist_2000_16pfts_c240908.nc' failed
svn: E175002: PUT request on '/!svn/txr/68496-1k8e/trunk/inputdata/lnd/clm2/surfdata_esmf/ctsm5.3.0/surfdata_mpasa3p75_hist_2000_16pfts_c240908.nc' failed
svn: E175002: Unexpected HTTP status 413 'Request Entity Too Large' on '/!svn/txr/68507-1k8q/trunk/inputdata/lnd/clm2/surfdata_esmf/ctsm5.3.0/surfdata_ne120np4.pg3_hist_1850_78pfts_c240908.nc'
svn: E175002: PUT request on '/!svn/txr/68507-1k8r/trunk/inputdata/lnd/clm2/surfdata_esmf/ctsm5.3.0/surfdata_ne120np4.pg3_hist_2000_78pfts_c240908.nc' failed
```

_Originally posted by @slevis-lmwg in https://github.com/ESCOMP/CTSM/issues/2500#issuecomment-2339266811_

Definition of done:

- [x] Have a backup copy of these files on Izumi
- [ ] This list of files is imported into the new system
- [ ] Remove the extra copy on Izumi

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.