Climate-REF / Climate-REF/climate-ref

Require a re-mint when diagnostic_version increases

Open
#855 0 comments 0 reactions 1 assignee Claimed by @lewisjared View on GitHub
Dominant language
Python
Stars
26
Forks
14
Avg merge
1d 10h
Merged PRs (30d)
36

Description

## Summary

Prevent a version-only `manifest.diagnostic_version` update from allowing a stale regression baseline to pass the CI coupling gate.

## Rationale

The current gate can accept this sequence:

1. `Diagnostic.version` increases.
2. A pull request updates only `manifest.diagnostic_version` to the new value.
3. `test_case_version`, `committed`, and `native` remain unchanged.
4. The gate returns `SKIP`.

This allows committed and native baseline data produced by the earlier diagnostic version to remain in use.

This behaviour predates #854. It is intentionally out of scope for that pull request.

## Required changes

Update the regression gate policy so that an increase in `diagnostic_version` requires a re-mint-authorised baseline update. A `test_case_version` increase is one possible authorisation mechanism.

Add regression coverage for a version-only manifest edit. The test must verify that the gate does not return `SKIP` when `diagnostic_version` increases without the required baseline update.

## Affected areas

- `packages/climate-ref-core/src/climate_ref_core/regression/gate.py`
- `packages/climate-ref-core/tests/unit/regression/test_gate.py`
- Related regression-gate documentation, if the policy changes documented behaviour.

## Acceptance criteria

- A `diagnostic_version` increase cannot pass with unchanged `test_case_version`, `committed`, and `native` baseline data.
- The gate emits a clear failure or required-action result for this case.
- Unit tests cover the version-only manifest update.
- Documentation describes the final policy.

## Backlinks

- Pull request: https://github.com/Climate-REF/climate-ref/pull/854
- Review discussion: https://github.com/Climate-REF/climate-ref/pull/854#discussion_r3732318995

Requested by @lewisjared.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.