memory error on NIRD when processing daily data for NorESM
- Dominant language
- Python
- Stars
- 1
- Forks
- 8
- Avg merge
- 1d 16h
- Merged PRs (30d)
- 15
Description
I am using the validation-script branch on nird.
Monthly processing using the following worked
`./cmor_driver.py --realm atmos --tsdir ~/test_cmor/n1850.ne16pg3_tn14.noresm3_0_beta17.Run12-1.restartfromold.2026-05-21/atm/timeseries --outdir ~/test_cmor/n1850.ne16pg3_tn14.noresm3_0_beta17.Run12-1.restartfromold.2026-05-21/cmor --workers 1 --resolution ne16 --model noresm `
However, following gives rise to a memory error
`./cmor_driver.py --realm atmos --tsdir ~/test_cmor/n1850.ne16pg3_tn14.noresm3_0_beta17.Run12-1.restartfromold.2026-05-21/atm/timeseries --outdir ~/test_cmor/n1850.ne16pg3_tn14.noresm3_0_beta17.Run12-1.restartfromold.2026-05-21/cmor --workers 1 --resolution ne16 --model noresm --frequency day`
ERROR:cmip7_prep.cmor_driver:Exception while processing zg_tavg-p39-hy-air with dims ['time', 'plev', 'lat']: MemoryError((58, 12, 96, 144), dtype('float64'))
zg is the last field to be processed - but there is a vertical interpolation that is being done.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with cmor_driver.py and reproduce the daily command on NIRD, comparing it with the working monthly command. Trace processing of zg_tavg-p39-hy-air and its vertical interpolation; done means daily processing completes without the reported MemoryError.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- data-engineering
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100