tidyverts / tidyverts/fabletools
Extending methods for distributional accuracy scores?
@mitchelloharawild is already working on this.
Since Nov 23, 2021.
- Dominant language
- R
- Stars
- 98
- Forks
- 37
- PR merge metrics
- No merged PRs in 30d
Description
Amazing package here, thanks!
I really appreciate the design of extendable models which is so nicely described in your vignette.
It seems that it should also be possible to extend the set of scoring rules, in particular, for distributions, but I would find some guidance helpful. You are probably already familiar with the scoringRules package, https://cran.r-project.org/web/packages/scoringRules/index.html, which does a rather good job IMO of providing a good number of very computationally efficient implementations of scoring rules for distributions, and it would be lovely to see how we might best plug those existing methods into fabletools accuracy calculations.
In particular, I have found the strictly proper rule of Logarithmic score to be an appealing alternative to CRPS, especially under circumstances where underestimating the probability of a rare event can be catastrophic. I believe the implementations in scoringRules could potentially provide improved performance in scoring very large forecasts.
Thanks for considering!
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.