ESMCI / ESMCI/cmip7-prep

Memory and storage improve gen_timeseries by grabbing variables to make from yaml

Open
#90 6 comments 0 reactions 1 assignee Claimed by @AgentOxygen View on GitHub
enhancement
Dominant language
Python
Stars
1
Forks
8
Avg merge
1d 16h
Merged PRs (30d)
15

Description

Currently gen_timeseries.py generates timeseries for all variables that are present in the file-type it is asked to look at.

Not infrequently, at least for longer input runs, gen_timeseries.py fails with what looks like memory related issues, and it is also not ideal to generate all outputs under the sun for intermediate storage.

One part of the solution (which would be useful) could be to have gen_timeseries.py read in the corresponding yaml for the realm under consideration, and only generate timeseries for available variables listed in that yaml (subsetting variables more or less on opening of the file). This could also be used to find any fields that are instantaneous and make them from a different file type (hxi over hxa) as described in #79 .

To discuss further tomorrow with @AgentOxygen and @mvertens

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.