MetOffice / MetOffice/CMEW

Implement REF cataloguing (ingest) for CMIP and development runs

Open
#357 0 comments 0 reactions 0 assignees View on GitHub
rapid evaluation framework
Dominant language
Python
Stars
5
Forks
3
Avg merge
5d 6h
Merged PRs (30d)
8

Description

Implement the following REF ingest steps:
1. Ingest standardised runs
2. Ingest CMIP data from manage_cmip for the activities / models / experiments requested by the CMEW user
3. Combine catalogues ready for use in ref solve
(There is probably a technical question about whether these are separate apps or not, which we can defer to planning.)
Dependencies for this issue:

Acceptance criteria:
- Given CMEW configuration specifying at least one standardised model development run and at least one CMIP run
- When new catalogue task (or tasks) run within CMEW
- Then a combined catalogue is generated that is useable by ref solve (could be validated outside of CMEW if solve not yet implemented in that point and/or set up a test in CMEW)

Dependencies:
- We shouldn't start work on the ingest issue(s) until we are confident that we will use the catalogue, so e.g. pending resolution of solve failure (e.g. https://github.com/Climate-REF/climate-ref/issues/494).
- There is no dependency of ingest on configuration files
- REF does require its environment and creates environments for "providers" (ESMValTool, PMP, ...) - investigation ongoing in SSS repo

Contributor guide

Open the contributing guide

Research direction

Start with the CMEW configuration and the new catalogue task or tasks described in the acceptance criteria, then review Climate-REF issue 494 and the ongoing SSS repository investigation. The work is done when a configuration containing at least one standardised development run and one CMIP run produces a combined catalogue usable by ref solve; confirm this with a CMEW test or external validation if solve is not yet implemented.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.