Restore previously standardised data from MASS
- Dominant language
- Python
- Stars
- 5
- Forks
- 3
- Avg merge
- 5d 6h
- Merged PRs (30d)
- 8
Description
For large datasets (esp high resolution), the standardisation step is a significant cost. In many cases CMEW will make use of the same model development runs over and over again, so the standardisation cost will be duplicated.
User story:
- As a CMEW user
- I want CMEW to make use of previously standardised data
- So that I can get my results faster and save compute cost
Implementation ideas:
- CDDS can (optionally?) archive standardised runs back to MASS
- #282 will enable us to switch off the CDDS extract step, so the main remaining gap after that will be to add capability to restore previously standardised data
Contributor guide
Research direction
Start by reading the CDDS extract work in #282 and tracing how standardised runs are archived to or restored from MASS. The work is done when CMEW can identify and reuse previously standardised data instead of repeating the standardisation step, while preserving the existing workflow behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- data
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100