CliMA / CliMA/ClimaAnalysis.jl
Computing bias/rmse/leaderboard from OutputVars
- Dominant language
- Julia
- Stars
- 12
- Forks
- 5
- Avg merge
- 7d 2h
- Merged PRs (30d)
- 2
Description
:warning: This SDI is actively being written
# The Climate Modeling Alliance
## Software Design Issue 📜
### Purpose
[ClimaCoupler](https://github.com/CliMA/ClimaCoupler.jl/tree/main/experiments/ClimaEarth/user_io/leaderboard) implements a "Leaderboard" module. Fundamentally, this module does two things:
1. Allow users to specify observational data and compute bias and RMSE between simulated data and observations
2. Allow users to specify RMSEs for other models and compare how our model performs
While the module works, there are some drawbacks, mostly stemming from the fact that the code is embedded in ClimaCoupler
- it clunky to use if one has already some output
- small modifications require going through the entire ClimaCoupler CI (which takes >1 h)
- it cannot be used by other packages
With this SDI, we move the module from ClimaCoupler to ClimaAnalysis. In the process, we update it to make use of newer features recently introduced in ClimaAnalysis (that will lead to simpler code).
### Cost/Benefits/Risks
Benefits include:
- Fixing the problems above
- Allowing ClimaLand to directly use this code for comparing against observations
- Extracting some code that is useful in itself (e.g., computing integrals of `OutputVar`s)
The logic of the code is already implemented and working, so the main cost is developer time.
### People and Personnel
- Lead: @ph-kev
- Collaborators: @Sbozzolo
- Reviewers: @Sbozzolo
### Components
### Inputs
### Results and Deliverables
### Scope of Work
At higher level, there are two main components:
1. computing/plotting bias/rmse
2. computing/plotting leaderboard
```[tasklist]
### Bias/RMSE
- [ ] https://github.com/CliMA/ClimaAnalysis.jl/issues/59
- [ ] https://github.com/CliMA/ClimaAnalysis.jl/issues/62
- [ ] https://github.com/CliMA/ClimaAnalysis.jl/issues/68
- [ ] https://github.com/CliMA/ClimaAnalysis.jl/issues/65
- [ ] https://github.com/CliMA/ClimaAnalysis.jl/issues/75
- [ ] https://github.com/CliMA/ClimaAnalysis.jl/issues/84
- [ ] https://github.com/CliMA/ClimaAnalysis.jl/issues/86
- [ ] https://github.com/CliMA/ClimaAnalysis.jl/issues/78
- [x] Provide functions for unit conversions
- [ ] https://github.com/CliMA/ClimaAnalysis.jl/issues/88
- [ ] https://github.com/CliMA/ClimaAnalysis.jl/issues/89
- [ ] https://github.com/CliMA/ClimaAnalysis.jl/issues/90
- [ ] https://github.com/CliMA/ClimaAnalysis.jl/issues/94
- [ ] https://github.com/CliMA/ClimaAnalysis.jl/issues/95
- [ ] https://github.com/CliMA/ClimaAnalysis.jl/issues/97
- [ ] https://github.com/CliMA/ClimaAnalysis.jl/issues/99
- [ ] https://github.com/CliMA/ClimaAnalysis.jl/issues/98
- [x] Refactor leaderboard code in ClimaCoupler.jl
- [ ] Add leaderboard in ClimaLand.jl
```
Once this is completed, we will be able to directly compute bias and mse from two OutputVars. Packages like ClimaCoupler will load simulation data and observation data as OutputVars.
### SDI Revision Log
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.