MetOffice / MetOffice/CMEW

Restore previously standardised data from MASS

Open
#352 0 comments 0 reactions 0 assignees View on GitHub
enhancement standardise
Dominant language
Python
Stars
5
Forks
3
Avg merge
5d 6h
Merged PRs (30d)
8

Description

For large datasets (esp high resolution), the standardisation step is a significant cost. In many cases CMEW will make use of the same model development runs over and over again, so the standardisation cost will be duplicated.

User story:
- As a CMEW user
- I want CMEW to make use of previously standardised data
- So that I can get my results faster and save compute cost

Implementation ideas:
- CDDS can (optionally?) archive standardised runs back to MASS
- #282 will enable us to switch off the CDDS extract step, so the main remaining gap after that will be to add capability to restore previously standardised data

Contributor guide

Open the contributing guide

Research direction

Start by reading the CDDS extract work in #282 and tracing how standardised runs are archived to or restored from MASS. The work is done when CMEW can identify and reuse previously standardised data instead of repeating the standardisation step, while preserving the existing workflow behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.