Climate-REF / Climate-REF/climate-ref

cloud-scatterplots-cli-ta fails on hybrid-coordinate models: CMOR check rejects duplicate air_pressure standard_name

Open
#841 3 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
26
Forks
14
Avg merge
1d 10h
Merged PRs (30d)
36

Description

`esmvaltool/cloud-scatterplots-cli-ta` fails for roughly three quarters of the models it is solved for, and the split is systematic by model rather than by member or by data gap.

## Observed

From a full CMIP6 historical solve (11,170 datasets):

| outcome | models |
| --- | --- |
| failed (16 executions) | ACCESS-ESM1-5, CMCC-ESM2, CMCC-CM2-SR5, CanESM5, CanESM5-1 |
| succeeded (6 executions) | AWI-ESM-1-1-LR, BCC-ESM1, CESM2-FV2, CESM2-WACCM-FV2 |

Every member of a failing model fails and every member of a succeeding model succeeds, which points at a model property rather than at missing files.

## Error

```
ValueError: Can not concatenate cubes: failed to concatenate into a single cube.
Auxiliary coordinates are unequal for phenomenon `mass_fraction_of_cloud_ice_in_air`:
a: _CoordAndDims(coord=+bounds shape(38, 145, 192)>, dims=(1, 2, 3))
b: _CoordAndDims(coord=+bounds shape(38, 145, 192)>, dims=(1, 2, 3))
```

The resulting cube in the case I inspected is on `atmosphere_hybrid_height_coordinate`. The two `air_pressure` aux coords have identical shape but unequal values, which is expected: on a hybrid vertical coordinate the derived pressure is time-varying, so two time chunks genuinely differ and iris refuses to concatenate them.

The failure propagates out as `subprocess.CalledProcessError` from the micromamba run, so the execution is marked failed with no metrics.

I have only confirmed the vertical coordinate for one of the failing models, so treat the common mechanism across all five as likely rather than established.

## Why it matters

At AFT scale this silently drops a cloud diagnostic for a large share of the ensemble, and the models affected include several widely used ones. Dropping the derived `air_pressure` aux coord before concatenation, or concatenating on the model levels and deriving pressure afterwards, are the usual remedies.

Seen on climate-ref v0.16.2, climate-ref-esmvaltool 0.16.2.

Contributor guide

Open the contributing guide

Research direction

Start with the cloud-scatterplots-cli-ta entry point and reproduce the CMOR concatenation failure for a failing hybrid-coordinate model. Trace how the time chunks and derived air_pressure auxiliary coordinates are handled before concatenation, then verify that the affected models complete and produce metrics without the duplicate-coordinate error.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.