Climate-REF / Climate-REF/climate-ref

Feedback as a non-ESGF user: discoverability of diagnostics, opaque provider config, and internet reliance

Open
#469 3 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
26
Forks
14
Avg merge
1d 10h
Merged PRs (30d)
36

Description

Hi all,

I have recently been diving a bit more into the REF (sorry it has been a while, I had to switch back to other projects for a bit). What follows is feedback purely from the perspective of someone using REF as a tool for running evaluation **outside of the ESGF workflow**, so it might or might not apply broadly.

### 1. Hard to discover available diagnostics

At the moment it is a bit difficult to figure out *what diagnostics are actually available to me*. I am constantly jumping between:

* the diagnostic providers’ documentation (e.g. ESMValTool recipes, PMP docs, etc.)
* the data I have already ingested into REF
* the REF documentation

It would help a lot to have something like:

```
ref diagnostic --list
```

Ideally showing the *available* diagnostics based on what I have ingested. Without this, discoverability feels guess-and-check, and I am not sure this will be practical in normal usage.

### 2. Provider configuration is opaque in REF

When using REF, the provider environment (e.g. ESMValTool) is installed, but it is not clear **how it is configured** inside REF. On HPC systems this will be problematic, especially for centres that want to:

* use their own data pools
* rely on site-specific configuration files

Since we do not see the config, it’s not obvious how to override it.

**Suggestion:** allow optional additional configuration paths in `ref.toml` that point to local provider settings (like ESMValTool `config-user.yml`). Maybe something like:

```toml
[providers.esmvaltool]
config = "/path/to/esmvaltool/config-user.yml"
```

Having some way to explicitly manage this would make HPC deployment much more realistic.

### 3. Internet reliance for retrieving diagnostics

I raised this in a previous issue #468 , but I want to reinforce it: **REF should not rely on internet access** to download diagnostics, documentation, or recipes. Many HPC centres restrict external network access, or only allow it on very small queues with limited resources. The current approach is going to be a blocker for many sites.

I know there is no “easy” solution here, but for HPC use it might be essential to:

* vendor or mirror diagnostic metadata
* ship diagnostics with REF releases
* provide offline plugin bundles

Anything that removes network reliance would help.

---

Happy to clarify any of the points above. Thanks again for all the work — really impressive progress overall!

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.