Climate-REF / Climate-REF/climate-ref

Diagnostic cloud-scatterplots-cli-ta expects mip=realm for OBS

Open
#788 4 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
Python
Stars
26
Forks
14
Avg merge
1d 10h
Merged PRs (30d)
36

Description

## Describe the bug

I get the following error message (below) when trying to run the `cloud-scatterplots-cli-ta` diagnostic with a CMIP7 model, where the relevant error message (one of several) is:
```
2026-07-03 22:57:48,876 UTC [3019873] ERROR No files were found for Dataset:
{'dataset': 'ISCCP-FH',
'project': 'OBS',
'mip': 'atmos',
'short_name': 'rsutcs',
'alias': 'OBS_ISCCP-FH',
'derive': True,
'frequency': '',
'long_name': 'TOA Outgoing Clear-Sky Shortwave Radiation',
'modeling_realm': ['atmos'],
'original_short_name': 'rsutcs',
'preprocessor': 'zonal_mean',
'recipe_dataset_index': 3,
'standard_name': 'toa_outgoing_shortwave_flux_assuming_clear_sky',
'tier': 2,
'timerange': '1996/2016',
'type': 'sat',
'units': 'W m-2',
'version': 'v0'}
session: 'recipe_20260703_225747',
using data sources:
- data source: LocalDataSource(name='local', project='OBS', priority=1, rootpath=PosixPath('/fs/site7/eccc/crd/cccma/users/scrd115/REF_v4/climate-ref/.ref/cache/esmvaltool/datasets/ESMValTool/OBS'), dirname_template='Tier{tier}/{dataset}', filename_template='{project}_{dataset}_{type}_{version}_{mip}_{short_name}[_.]*nc', ignore_warnings=[])
message: No files found matching glob pattern /fs/site7/eccc/crd/cccma/users/scrd115/REF_v4/climate-ref/.ref/cache/esmvaltool/datasets/ESMValTool/OBS/Tier2/ISCCP-FH/OBS_ISCCP-FH_sat_v0_atmos_rsutcs[_.]*nc
```

My interpretation of the filename, having `_atmos_` (instead of `_Amon_` where the diagnostic is run for a CMIP6 model), is that this diagnostic expects a different `mip` facet value when comparing to CMIP7? this is a problem, as the data provided (via the fetch function) has the filename structure where `mip=`

**out.log** contents attached

## System

- OS: Linux
- Python version: 3.11.9

[out.log](https://github.com/user-attachments/files/29650772/out.log)

Contributor guide

Open the contributing guide

Research direction

Start with the cloud-scatterplots-cli-ta diagnostic and the attached out.log, then trace how its OBS dataset metadata sets the mip facet. Compare the requested atmos path with the fetch function's CMOR6 table_id filename structure; done means the diagnostic locates the provided OBS files for CMIP7 without breaking the CMIP6 case.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data-engineering
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
52/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.