Climate-REF / Climate-REF/climate-ref
Feedback as a non-ESGF user: discoverability of diagnostics, opaque provider config, and internet reliance
- Dominant language
- Python
- Stars
- 26
- Forks
- 14
- Avg merge
- 1d 10h
- Merged PRs (30d)
- 36
Description
Hi all,
I have recently been diving a bit more into the REF (sorry it has been a while, I had to switch back to other projects for a bit). What follows is feedback purely from the perspective of someone using REF as a tool for running evaluation **outside of the ESGF workflow**, so it might or might not apply broadly.
### 1. Hard to discover available diagnostics
At the moment it is a bit difficult to figure out *what diagnostics are actually available to me*. I am constantly jumping between:
* the diagnostic providers’ documentation (e.g. ESMValTool recipes, PMP docs, etc.)
* the data I have already ingested into REF
* the REF documentation
It would help a lot to have something like:
```
ref diagnostic --list
```
Ideally showing the *available* diagnostics based on what I have ingested. Without this, discoverability feels guess-and-check, and I am not sure this will be practical in normal usage.
### 2. Provider configuration is opaque in REF
When using REF, the provider environment (e.g. ESMValTool) is installed, but it is not clear **how it is configured** inside REF. On HPC systems this will be problematic, especially for centres that want to:
* use their own data pools
* rely on site-specific configuration files
Since we do not see the config, it’s not obvious how to override it.
**Suggestion:** allow optional additional configuration paths in `ref.toml` that point to local provider settings (like ESMValTool `config-user.yml`). Maybe something like:
```toml
[providers.esmvaltool]
config = "/path/to/esmvaltool/config-user.yml"
```
Having some way to explicitly manage this would make HPC deployment much more realistic.
### 3. Internet reliance for retrieving diagnostics
I raised this in a previous issue #468 , but I want to reinforce it: **REF should not rely on internet access** to download diagnostics, documentation, or recipes. Many HPC centres restrict external network access, or only allow it on very small queues with limited resources. The current approach is going to be a blocker for many sites.
I know there is no “easy” solution here, but for HPC use it might be essential to:
* vendor or mirror diagnostic metadata
* ship diagnostics with REF releases
* provide offline plugin bundles
Anything that removes network reliance would help.
---
Happy to clarify any of the points above. Thanks again for all the work — really impressive progress overall!
Contributor guide
Assessment
This issue has not been assessed yet.