CliMA / CliMA/ClimaAnalysis.jl

Computing bias/rmse/leaderboard from OutputVars

Open
#58 0 comments 0 reactions 1 assignee Claimed by @ph-kev View on GitHub
Dominant language
Julia
Stars
12
Forks
5
Avg merge
7d 2h
Merged PRs (30d)
2

Description

:warning: This SDI is actively being written

# The Climate Modeling Alliance
## Software Design Issue 📜
### Purpose
[ClimaCoupler](https://github.com/CliMA/ClimaCoupler.jl/tree/main/experiments/ClimaEarth/user_io/leaderboard) implements a "Leaderboard" module. Fundamentally, this module does two things:
1. Allow users to specify observational data and compute bias and RMSE between simulated data and observations
2. Allow users to specify RMSEs for other models and compare how our model performs

While the module works, there are some drawbacks, mostly stemming from the fact that the code is embedded in ClimaCoupler
- it clunky to use if one has already some output
- small modifications require going through the entire ClimaCoupler CI (which takes >1 h)
- it cannot be used by other packages

With this SDI, we move the module from ClimaCoupler to ClimaAnalysis. In the process, we update it to make use of newer features recently introduced in ClimaAnalysis (that will lead to simpler code).

### Cost/Benefits/Risks
Benefits include:
- Fixing the problems above
- Allowing ClimaLand to directly use this code for comparing against observations
- Extracting some code that is useful in itself (e.g., computing integrals of `OutputVar`s)

The logic of the code is already implemented and working, so the main cost is developer time.

### People and Personnel
- Lead: @ph-kev
- Collaborators: @Sbozzolo
- Reviewers: @Sbozzolo

### Components

### Inputs

### Results and Deliverables

### Scope of Work

At higher level, there are two main components:
1. computing/plotting bias/rmse
2. computing/plotting leaderboard

```[tasklist]
### Bias/RMSE
- [ ] https://github.com/CliMA/ClimaAnalysis.jl/issues/59
- [ ] https://github.com/CliMA/ClimaAnalysis.jl/issues/62
- [ ] https://github.com/CliMA/ClimaAnalysis.jl/issues/68
- [ ] https://github.com/CliMA/ClimaAnalysis.jl/issues/65
- [ ] https://github.com/CliMA/ClimaAnalysis.jl/issues/75
- [ ] https://github.com/CliMA/ClimaAnalysis.jl/issues/84
- [ ] https://github.com/CliMA/ClimaAnalysis.jl/issues/86
- [ ] https://github.com/CliMA/ClimaAnalysis.jl/issues/78
- [x] Provide functions for unit conversions
- [ ] https://github.com/CliMA/ClimaAnalysis.jl/issues/88
- [ ] https://github.com/CliMA/ClimaAnalysis.jl/issues/89
- [ ] https://github.com/CliMA/ClimaAnalysis.jl/issues/90
- [ ] https://github.com/CliMA/ClimaAnalysis.jl/issues/94
- [ ] https://github.com/CliMA/ClimaAnalysis.jl/issues/95
- [ ] https://github.com/CliMA/ClimaAnalysis.jl/issues/97
- [ ] https://github.com/CliMA/ClimaAnalysis.jl/issues/99
- [ ] https://github.com/CliMA/ClimaAnalysis.jl/issues/98
- [x] Refactor leaderboard code in ClimaCoupler.jl
- [ ] Add leaderboard in ClimaLand.jl
```

Once this is completed, we will be able to directly compute bias and mse from two OutputVars. Packages like ClimaCoupler will load simulation data and observation data as OutputVars.

### SDI Revision Log

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.