Climate-REF / Climate-REF/climate-ref

Committed output.json references index.html / cmec_output.log that are never produced (absent from manifest native)

Open
#763 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
26
Forks
14
Avg merge
1d 10h
Merged PRs (30d)
36

Description

## Summary

Committed regression `output.json` bundles reference two artefacts — `index.html` (top-level `index`) and `cmec_output.log` (`provenance.log`) — that the diagnostics never actually produce, so they are absent from the test case's `manifest.json` `native` block. The committed bundle and the native manifest are therefore out of sync: a consumer reading `output.json` would expect those files to exist, but `ref test-cases replay` has nothing to materialise for them.

Surfaced in review of #762 on `extratropical-modes-of-variability-nam/cmip7`:

> output.json still references index.html and cmec_output.log, but neither artefact is listed under native here. Add them to native, or remove those references from output.json.

## Root cause

`OutputBundle.create_template()` seeds every CMEC output bundle with hardcoded defaults:

`packages/climate-ref-core/src/climate_ref_core/pycmec/output.py:202-218`
```python
return {
OutputCV.INDEX.value: "index.html",
OutputCV.PROVENANCE.value: {
...
OutputCV.LOG.value: "cmec_output.log",
},
...
}
```

These defaults persist into the committed `output.json` even when the diagnostic writes neither file.

## Scope (committed baselines on `main`, 43 cases scanned)

| Artefact | Referenced by | Present in `native` |
|---|---|---|
| `cmec_output.log` | all 43 | **0** — never generated by any provider |
| `index.html` | all 43 | ilamb 20/20; **pmp 0/21, example 0/2** |

So `cmec_output.log` is a universal dangling reference, and `index.html` dangles for the 23 pmp + example cases (ilamb genuinely produces it). esmvaltool baselines (pending in #760) most likely follow the pmp/example pattern.

Contributor guide

Open the contributing guide

Research direction

Read `packages/climate-ref-core/src/climate_ref_core/pycmec/output.py` at `OutputBundle.create_template()` and inspect the committed `output.json` files against each case's `manifest.json` `native` block. Run `ref test-cases replay` to check what is materialised; done means committed bundles no longer reference artefacts that the diagnostics do not produce.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
testing
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
68/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.