easystats / easystats/performance

`r2_nakagawa()` is **NOT** location invariant when COV is not modelled (and is wrong?)

Open
#428 4 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

3 investigators :grey_question::question: Bug :bug:
Dominant language
R
Stars
1.2k
Forks
109
Avg merge
6h 34m
Merged PRs (30d)
8

Description

This is some weird stuff.

library(lme4)
#> Loading required package: Matrix
library(performance)

r2_nakagawa2 <- function(model) {
  y <- insight::get_response(model)
  c(
    Cond. = cor(predict(model, re.form = NULL), y)^2,
    Marg. = cor(predict(model, re.form = NA), y)^2
  )
}


# With COV =================================
m1 <- lmer(Reaction ~ Days + (Days|Subject),
          data = sleepstudy)


# Change location -------
sleepstudy$Days0 <- sleepstudy$Days - 10
m2 <- lmer(Reaction ~ Days0 + (Days0|Subject),
          data = sleepstudy)
#> Warning in checkConv(attr(opt, "derivs"), opt$par, ctrl = control$checkConv, :
#> Model failed to converge with max|grad| = 0.00422952 (tol = 0.002, component 1)

r2_nakagawa(m1)
#> # R2 for Mixed Models
#> 
#>   Conditional R2: 0.799
#>      Marginal R2: 0.279
r2_nakagawa(m2) # same
#> # R2 for Mixed Models
#> 
#>   Conditional R2: 0.799
#>      Marginal R2: 0.279

# Validate:
r2_nakagawa2(m1)
#>     Cond.     Marg. 
#> 0.8271703 0.2864714
r2_nakagawa2(m2)
#>     Cond.     Marg. 
#> 0.8271702 0.2864714


# Without COV =================================
m1b <- lmer(Reaction ~ Days + (Days||Subject),
           data = sleepstudy)


# Change location -------
m2b <- lmer(Reaction ~ Days0 + (Days0||Subject),
           data = sleepstudy)


r2_nakagawa(m1b) # different
#> # R2 for Mixed Models
#> 
#>   Conditional R2: 0.702
#>      Marginal R2: 0.415
r2_nakagawa(m2b) # even more different
#> # R2 for Mixed Models
#> 
#>   Conditional R2: 0.864
#>      Marginal R2: 0.183

r2_nakagawa2(m1b) # same as with cov
#>     Cond.     Marg. 
#> 0.8279419 0.2864714
r2_nakagawa2(m2b) # same as with cov
#>     Cond.     Marg. 
#> 0.8244819 0.2864714

Created on 2022-06-03 by the reprex package (v2.0.1)

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the r2_nakagawa() entry point and reproduce the lme4 sleepstudy examples in the issue, comparing models with and without covariance and shifted Days values. Determine the intended location-invariant result for models where COV is not modelled; done means the behavior is corrected and covered by an appropriate regression check.

Written by the indexing model from the issue text.

Assessment

Tech stack
r
Domain
data
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.