MetOffice / MetOffice/CMEW

Make a local cache of extracted raw data for running with extract switched off

Open
#355 1 comment 0 reactions 0 assignees View on GitHub
configure standardise
Dominant language
Python
Stars
5
Forks
3
Avg merge
5d 6h
Merged PRs (30d)
8

Description

Once #282 is complete, CMEW will be able to use previously extracted data and skip extracting data from the MASS archive. For this to work, a user needs to point CMEW to previously extracted data, and this would be easier if we created a central cache for a few model runs, sufficient for the default runs in the standard configuration of CMEW.

Acceptance criteria:
- Given the standard configuration of CMEW, but with extract switched off and `RAW_DATA_DIR` pointing to the new cache
- When CMEW is run
- It works!

Contributor guide

Open the contributing guide

Research direction

Start by examining CMEW's standard configuration, the extraction workflow, and how `RAW_DATA_DIR` is consumed; issue #282 describes the prerequisite behavior. Done means the standard configuration runs successfully with extraction disabled and `RAW_DATA_DIR` pointed at the new cache.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data-engineering
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.