MetOffice / MetOffice/CDDS

Allow fixed fields to be processed and archived alongside other data

Open
#541 0 comments 0 reactions 0 assignees View on GitHub
migrated
Dominant language
Python
Stars
8
Forks
7
Avg merge
1d 15h
Merged PRs (30d)
23

Description

Migrated Jira Issue CDDSO-92
Jira creation date 25/04/2022 17:25
-------
At present the inclusion of fixed fields in a round of processing is a very manual process. It would be useful to consider the changes needed to enable this. My run through from arise has the following features;

Extract -- successfully did nothing

Configure – had to add an ancilliary file to enable fx

Convert – not designed to allow single cycle ancilliaries, would need “ancil” stream to be added

QC – no time axis found in files so failed

Store – no time bounds in file names so failed.

Historically we haven’t bothered producing the 9 data sets for anything other than the CMIP6 piControl runs which I did by hand. It would be useful to work out how to adapt to allow for this
-------

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by tracing how fixed fields and ancillary streams move through Extract, Configure, Convert, QC, and Store, using the CMIP6 piControl workflow as the comparison case. Determine what changes are needed for fixed fields to pass each stage and be archived with other data; done means the full processing path works without manual intervention.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data-engineering
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.