MetOffice / MetOffice/CDDS

Bug in `calc_fgdms` when doing <yearly processing

Open
#960 0 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
Python
Stars
8
Forks
7
Avg merge
1d 15h
Merged PRs (30d)
23

Description

I'm guessing this is low priority as it feels like a rare edge case, so this is mostly just to record some findings if needed.

I ran a request with < 1year start and end date to save some time when iterating development and `fgdms_tavg-u-hxy-sea` was failing with this error.

```
*** Starting conversions ***
Loading data for "CMIP7_ocnBgchem.json: fgdms_tavg-u-hxy-sea"
Unable to produce MIP requested variable "fgdms_tavg-u-hxy-sea" for "CMIP7_ocnBgchem": Field and Land Fraction dimensions differ
Unable to produce MIP requested variable "fgdms_tavg-u-hxy-sea" for "CMIP7_ocnBgchem": Field and Land Fraction dimensions differ
Field and Land Fraction dimensions differ
Traceback (most recent call last):
File "(redacted)/CDDS/mip_convert/mip_convert/request.py", line 124, in convert
produce_mip_requested_variable(variable_name, stream_id, substream, mip_table, user_config,
File "(redacted)/729_collect_ukcm2_ll_ancils/CDDS/mip_convert/mip_convert/requested_variables.py", line 132, in produce_mip_requested_variable
time_slice.process()
File "(redacted)29_collect_ukcm2_ll_ancils/CDDS/mip_convert/mip_convert/new_variable.py", line 361, in process
self._apply_expression()
File "(redacted)t_ukcm2_ll_ancils/CDDS/mip_convert/mip_convert/new_variable.py", line 552, in _apply_expression
self.cube = plugin.evaluate_expression(expression, self.input_variables)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/(redacted)ncils/CDDS/mip_convert/mip_convert/plugins/ukcm2/ukcm2_plugin.py", line 40, in evaluate_expression
return eval(expression)
^^^^^^^^^^^^^^^^
File "", line 1, in
File "(redacted)ds/729_collect_ukcm2_ll_ancils/CDDS/mip_convert/mip_convert/plugins/base/data/processors.py", line 1919, in calc_fgdms
raise RuntimeError('Field and Land Fraction dimensions differ')
RuntimeError: Field and Land Fraction dimensions differ`
```
When run >year start and end date it produces fine.

I think it's due to the indexing going on here:
https://github.com/MetOffice/CDDS/blob/6a4a7217f5a60d4a5e8d24a00549da1be9fd175f/mip_convert/mip_convert/plugins/base/data/processors.py#L1916-L1918

Here are the input cubes (before that indexing line) if

Contributor guide

No contributing guide indexed for this repository

Research direction

Start in mip_convert/mip_convert/plugins/base/data/processors.py around lines 1916-1918 and reproduce calc_fgdms with the under-one-year request. Compare the reported cube and land_fraction shapes with the five-year case, then verify that fgdms_tavg-u-hxy-sea completes without the dimension error for both request lengths.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data-engineering
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
66/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.