Bug in `calc_fgdms` when doing <yearly processing
- Dominant language
- Python
- Stars
- 8
- Forks
- 7
- Avg merge
- 1d 15h
- Merged PRs (30d)
- 23
Description
I'm guessing this is low priority as it feels like a rare edge case, so this is mostly just to record some findings if needed.
I ran a request with < 1year start and end date to save some time when iterating development and `fgdms_tavg-u-hxy-sea` was failing with this error.
```
*** Starting conversions ***
Loading data for "CMIP7_ocnBgchem.json: fgdms_tavg-u-hxy-sea"
Unable to produce MIP requested variable "fgdms_tavg-u-hxy-sea" for "CMIP7_ocnBgchem": Field and Land Fraction dimensions differ
Unable to produce MIP requested variable "fgdms_tavg-u-hxy-sea" for "CMIP7_ocnBgchem": Field and Land Fraction dimensions differ
Field and Land Fraction dimensions differ
Traceback (most recent call last):
File "(redacted)/CDDS/mip_convert/mip_convert/request.py", line 124, in convert
produce_mip_requested_variable(variable_name, stream_id, substream, mip_table, user_config,
File "(redacted)/729_collect_ukcm2_ll_ancils/CDDS/mip_convert/mip_convert/requested_variables.py", line 132, in produce_mip_requested_variable
time_slice.process()
File "(redacted)29_collect_ukcm2_ll_ancils/CDDS/mip_convert/mip_convert/new_variable.py", line 361, in process
self._apply_expression()
File "(redacted)t_ukcm2_ll_ancils/CDDS/mip_convert/mip_convert/new_variable.py", line 552, in _apply_expression
self.cube = plugin.evaluate_expression(expression, self.input_variables)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/(redacted)ncils/CDDS/mip_convert/mip_convert/plugins/ukcm2/ukcm2_plugin.py", line 40, in evaluate_expression
return eval(expression)
^^^^^^^^^^^^^^^^
File "", line 1, in
File "(redacted)ds/729_collect_ukcm2_ll_ancils/CDDS/mip_convert/mip_convert/plugins/base/data/processors.py", line 1919, in calc_fgdms
raise RuntimeError('Field and Land Fraction dimensions differ')
RuntimeError: Field and Land Fraction dimensions differ`
```
When run >year start and end date it produces fine.
I think it's due to the indexing going on here:
https://github.com/MetOffice/CDDS/blob/6a4a7217f5a60d4a5e8d24a00549da1be9fd175f/mip_convert/mip_convert/plugins/base/data/processors.py#L1916-L1918
Here are the input cubes (before that indexing line) if
Contributor guide
No contributing guide indexed for this repository
Research direction
Start in mip_convert/mip_convert/plugins/base/data/processors.py around lines 1916-1918 and reproduce calc_fgdms with the under-one-year request. Compare the reported cube and land_fraction shapes with the five-year case, then verify that fgdms_tavg-u-hxy-sea completes without the dimension error for both request lengths.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- data-engineering
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 66/100