MetOffice / MetOffice/CDDS

Make QC fail upon error parsing dataset

Open Beginner friendly
#932 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
8
Forks
7
Avg merge
1d 15h
Merged PRs (30d)
23

Description

https://github.com/MetOffice/CDDS/blob/a8b74e554d307965b0f93e31b3b3e7a251498d02/cdds/cdds/qc/dataset/cmip7.py#L144-L145

When QC experiences an error parsing a dataset (i.e. there is a missing attribute that causes a failure to aggregate files), functionality should be put in place so that QC fails rather than silently continuing. This will avoid issues with empty approved variable lists being parsed to transfer which then fails causing misleading error messages.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start at cdds/cdds/qc/dataset/cmip7.py lines 144-145 and inspect how dataset parsing errors are handled during file aggregation. Make QC fail when a missing attribute causes parsing to error, rather than continuing with an empty approved variable list; done means the misleading transfer failure is avoided.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data-engineering
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
68/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.