CFA Virtualisation using CMIP6 example data: Unable to aggregate
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 150
- Forks
- 23
- Avg merge
- 1d 11h
- Merged PRs (30d)
- 2
Description
Example CMIP6 data (JASMIN)
files = [
'/badc/cmip6/data/CMIP6/ScenarioMIP/CNRM-CERFACS/CNRM-ESM2-1/ssp119/r1i1p1f2/3hr/huss/gr/v20190328/huss_3hr_CNRM-ESM2-1_ssp119_r1i1p1f2_gr_201501010300-203501010000.nc',
'/badc/cmip6/data/CMIP6/ScenarioMIP/CNRM-CERFACS/CNRM-ESM2-1/ssp119/r1i1p1f2/3hr/huss/gr/v20190328/huss_3hr_CNRM-ESM2-1_ssp119_r1i1p1f2_gr_203501010300-205501010000.nc',
'/badc/cmip6/data/CMIP6/ScenarioMIP/CNRM-CERFACS/CNRM-ESM2-1/ssp119/r1i1p1f2/3hr/huss/gr/v20190328/huss_3hr_CNRM-ESM2-1_ssp119_r1i1p1f2_gr_205501010300-207501010000.nc',
'/badc/cmip6/data/CMIP6/ScenarioMIP/CNRM-CERFACS/CNRM-ESM2-1/ssp119/r1i1p1f2/3hr/huss/gr/v20190328/huss_3hr_CNRM-ESM2-1_ssp119_r1i1p1f2_gr_207501010300-209501010000.nc',
'/badc/cmip6/data/CMIP6/ScenarioMIP/CNRM-CERFACS/CNRM-ESM2-1/ssp119/r1i1p1f2/3hr/huss/gr/v20190328/huss_3hr_CNRM-ESM2-1_ssp119_r1i1p1f2_gr_209501010300-210101010000.nc'
]
Attempted to aggregate the first two example files (successful)
f = cf.read(files)
g = cf.aggregate(f[:2])
Normal cf.write functions properly here by creating a combined netCDF file of both files, but using with cfa=True results in one of the following, depending on if I take the whole of both Fields (116880 time steps):
RuntimeError: NetCDF: HDF error
or a subselection of the last 10 from file 1 and the first 10 from file 2 .
g = cf.aggregate([ f[0][-10:], f[1][:10] ])
File "/home/users/dwest77/Documents/cfa_python_dw/cf_dw/cf_python/cf/read_write/netcdf/netcdfwrite.py", line 106, in _write_as_cfa
raise ValueError(
ValueError: Can't write <CF Field: specific_humidity(time(20), latitude(128), longitude(256)) 1> as a CFA-netCDF aggregation variable. Consider setting cfa={'strict': False}
cf-python 3.16.2 (latest)
cfdm 1.11.1.0 (latest)
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reproducing the two examples with cf.read, cf.aggregate, and cf.write(cfa=True), then inspect cf/read_write/netcdf/netcdfwrite.py at _write_as_cfa. Compare the full and boundary-subset cases, and use the reported HDF error and strict-mode ValueError to trace the failing CFA-netCDF path. Done means the examples can be written with cfa=True without those failures.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- data
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100