easystats / easystats/performance
additonal adjusted R2 estimators
@bwiernik is already working on this.
Since Aug 9, 2021.
- Dominant language
- R
- Stars
- 1.2k
- Forks
- 109
- Avg merge
- 6h 34m
- Merged PRs (30d)
- 8
Description
Apparently, nowadays even Wikipedia points out the shortcomings of (adj.) R²:
Despite using unbiased estimators for the population variances of the error and the dependent variable, adjusted R2 is not an unbiased estimator of the population R2,[16] which results by using the population variances of the errors and the dependent variable instead of estimating them. Ingram Olkin and John W. Pratt derived the Minimum-variance unbiased estimator for the population R2,[17] which is known as Olkin-Pratt estimator. Comparisons of different approaches for adjusting R2 concluded that in most situations either an approximate version of the Olkin-Pratt estimator [16] or the exact Olkin-Pratt estimator [18] should be preferred over (Ezekiel) adjusted R2.
The German Wikipedia article even phrases it as (translated):
In practice, it is not recommended to use adj. R² for model selection, because the "penalty" for newly added explanatory variables seems too small. [...] For this reason, other criteria such as the AIC and the BIC have been developed for model selection, which also follow Occam's razor idea that a model should not be unnecessarily complex. [...] Numerous alternative estimators for R² in the population exist (see [51]). Of particular importance is the Olkin-Pratt estimator[52], since it is an unbiased estimator. In fact, it is the uniformly best unbiased estimator. Empirical comparisons of the different estimators logically conclude that in most cases the approximate[49] or the exact[50] Olkin-Pratt estimator should be used instead of the corrected coefficient of determination.
Luckily, Karch (2020) nicely compares the multiple measures and even published the {altR2} package which estimates several of them. Unfortunately, it appears to only work for lm() objects.
Question:
So my question is simply whether you have thought about implementing an Olkin-Pratt estimator to the {performance} package i.e. to model_performance() and if you have, I am genuinely curious why you decided against it.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.