easystats / easystats/performance

additonal adjusted R2 estimators

Open
#349 7 comments 0 reactions 1 assignee View on GitHub

@bwiernik is already working on this.

Since Aug 9, 2021.

Enhancement :boom:
Dominant language
R
Stars
1.2k
Forks
109
Avg merge
6h 34m
Merged PRs (30d)
8

Description

Apparently, nowadays even Wikipedia points out the shortcomings of (adj.) R²:

Despite using unbiased estimators for the population variances of the error and the dependent variable, adjusted R2 is not an unbiased estimator of the population R2,[16] which results by using the population variances of the errors and the dependent variable instead of estimating them. Ingram Olkin and John W. Pratt derived the Minimum-variance unbiased estimator for the population R2,[17] which is known as Olkin-Pratt estimator. Comparisons of different approaches for adjusting R2 concluded that in most situations either an approximate version of the Olkin-Pratt estimator [16] or the exact Olkin-Pratt estimator [18] should be preferred over (Ezekiel) adjusted R2.

The German Wikipedia article even phrases it as (translated):

In practice, it is not recommended to use adj. R² for model selection, because the "penalty" for newly added explanatory variables seems too small. [...] For this reason, other criteria such as the AIC and the BIC have been developed for model selection, which also follow Occam's razor idea that a model should not be unnecessarily complex. [...] Numerous alternative estimators for R² in the population exist (see [51]). Of particular importance is the Olkin-Pratt estimator[52], since it is an unbiased estimator. In fact, it is the uniformly best unbiased estimator. Empirical comparisons of the different estimators logically conclude that in most cases the approximate[49] or the exact[50] Olkin-Pratt estimator should be used instead of the corrected coefficient of determination.

Luckily, Karch (2020) nicely compares the multiple measures and even published the {altR2} package which estimates several of them. Unfortunately, it appears to only work for lm() objects.

Question:

So my question is simply whether you have thought about implementing an Olkin-Pratt estimator to the {performance} package i.e. to model_performance() and if you have, I am genuinely curious why you decided against it.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.