Climate-REF / Climate-REF/climate-ref
cloud-scatterplots-cli-ta fails on hybrid-coordinate models: CMOR check rejects duplicate air_pressure standard_name
- Dominant language
- Python
- Stars
- 26
- Forks
- 14
- Avg merge
- 1d 10h
- Merged PRs (30d)
- 36
Description
`esmvaltool/cloud-scatterplots-cli-ta` fails for roughly three quarters of the models it is solved for, and the split is systematic by model rather than by member or by data gap.
## Observed
From a full CMIP6 historical solve (11,170 datasets):
| outcome | models |
| --- | --- |
| failed (16 executions) | ACCESS-ESM1-5, CMCC-ESM2, CMCC-CM2-SR5, CanESM5, CanESM5-1 |
| succeeded (6 executions) | AWI-ESM-1-1-LR, BCC-ESM1, CESM2-FV2, CESM2-WACCM-FV2 |
Every member of a failing model fails and every member of a succeeding model succeeds, which points at a model property rather than at missing files.
## Error
```
ValueError: Can not concatenate cubes: failed to concatenate into a single cube.
Auxiliary coordinates are unequal for phenomenon `mass_fraction_of_cloud_ice_in_air`:
a: _CoordAndDims(coord=+bounds shape(38, 145, 192)>, dims=(1, 2, 3))
b: _CoordAndDims(coord=+bounds shape(38, 145, 192)>, dims=(1, 2, 3))
```
The resulting cube in the case I inspected is on `atmosphere_hybrid_height_coordinate`. The two `air_pressure` aux coords have identical shape but unequal values, which is expected: on a hybrid vertical coordinate the derived pressure is time-varying, so two time chunks genuinely differ and iris refuses to concatenate them.
The failure propagates out as `subprocess.CalledProcessError` from the micromamba run, so the execution is marked failed with no metrics.
I have only confirmed the vertical coordinate for one of the failing models, so treat the common mechanism across all five as likely rather than established.
## Why it matters
At AFT scale this silently drops a cloud diagnostic for a large share of the ensemble, and the models affected include several widely used ones. Dropping the derived `air_pressure` aux coord before concatenation, or concatenating on the model levels and deriving pressure afterwards, are the usual remedies.
Seen on climate-ref v0.16.2, climate-ref-esmvaltool 0.16.2.
Contributor guide
Research direction
Start with the cloud-scatterplots-cli-ta entry point and reproduce the CMOR concatenation failure for a failing hybrid-coordinate model. Trace how the time chunks and derived air_pressure auxiliary coordinates are handled before concatenation, then verify that the affected models complete and produce metrics without the duplicate-coordinate error.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- data
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100