easystats / easystats/modelbased

Support _metafor_ package (predictions and plots)

Open
#212 17 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Feature idea :fire:
Dominant language
R
Stars
263
Forks
22
Avg merge
1h 22m
Merged PRs (30d)
11

Description

Hi,

It seems that model_parameters() isn't calculating the correct point estimate (and 95% CI) for all subgroups in a metafor::rma() object. Below, I show that predict() and emmeans() yield the same results for all subgroups, while model_parameters() only recovers the "alternate" subgroup correctly.

# remotes::install_github("wviechtb/metafor") # install version 3.9-5
library(metafor)
library(emmeans)
library(parameters)

dat <- metadat::dat.bcg 

datRR <- metafor::escalc(measure="RR",
                         data=dat,
                         ai=tpos, bi=tneg, ci=cpos, di=cneg)

res <- metafor::rma(data=datRR,
                         method = "REML",
                         mods = ~ alloc,
                         yi = yi, vi = vi,
                         slab = author)
predict(res, newmods = rbind(c(1,0), # Random effects
                             c(0,0), # alternate
                             c(0,1)), # systematic
        digits=3) 
#> 
#>     pred    se  ci.lb  ci.ub  pi.lb pi.ub 
#> 1 -0.966 0.267 -1.490 -0.442 -2.255 0.324 
#> 2 -0.518 0.441 -1.383  0.347 -1.980 0.944 
#> 3 -0.429 0.345 -1.105  0.247 -1.788 0.930
gr <- metafor::emmprep(res) # version 3.9-5 required

emmeans::emmeans(gr, specs = ~alloc) 
#>  alloc      emmean    SE  df asymp.LCL asymp.UCL
#>  alternate  -0.518 0.441 Inf     -1.38     0.347
#>  random     -0.966 0.267 Inf     -1.49    -0.442
#>  systematic -0.429 0.345 Inf     -1.10     0.247
#> 
#> Results are given on the log (not the response) scale. 
#> Confidence level used: 0.95
parameters::model_parameters(res)
#> # Random Effects
#> 
#> Parameter         | Coefficient |   SE |         95% CI |      z |      p | Weight
#> ----------------------------------------------------------------------------------
#> Aronson           |       -0.89 | 0.57 | [-2.01,  0.23] |  -1.56 | 0.119  |   3.07
#> Ferguson & Simes  |       -1.59 | 0.44 | [-2.45, -0.72] |  -3.59 | < .001 |   5.14
#> Rosenthal et al.1 |       -1.35 | 0.64 | [-2.61, -0.08] |  -2.09 | 0.036  |   2.41
#> Hart & Sutherland |       -1.44 | 0.14 | [-1.72, -1.16] | -10.19 | < .001 |  49.97
#> Vandiviere et al  |       -1.62 | 0.47 | [-2.55, -0.70] |  -3.43 | < .001 |   4.48
#> TPT Madras        |        0.01 | 0.06 | [-0.11,  0.14] |   0.19 | 0.849  | 252.42
#> Coetzee & Berjak  |       -0.47 | 0.24 | [-0.94,  0.00] |  -1.98 | 0.048  |  17.72
#> Overall           |       -0.45 | 0.52 | [-1.46,  0.56] |  -0.87 | 0.385  |       
#> 
#> # alternate
#> 
#> Parameter            | Coefficient |   SE |         95% CI |     z |      p | Weight
#> ------------------------------------------------------------------------------------
#> Frimodt-Moller et al |       -0.22 | 0.23 | [-0.66,  0.23] | -0.96 | 0.336  |  19.53
#> Stein & Aronson      |       -0.79 | 0.08 | [-0.95, -0.62] | -9.46 | < .001 | 144.81
#> Overall              |       -0.52 | 0.44 | [-1.38,  0.35] | -1.17 | 0.240  |       
#> 
#> # systematic
#> 
#> Parameter          | Coefficient |   SE |         95% CI |     z |      p | Weight
#> ----------------------------------------------------------------------------------
#> Rosenthal et al.2  |       -1.37 | 0.27 | [-1.90, -0.84] | -5.07 | < .001 |  13.69
#> Comstock et al.1   |       -0.34 | 0.11 | [-0.56, -0.12] | -3.05 | 0.002  |  80.57
#> Comstock & Webster |        0.45 | 0.73 | [-0.98,  1.88] |  0.61 | 0.541  |   1.88
#> Comstock et al.2   |       -0.02 | 0.27 | [-0.54,  0.51] | -0.06 | 0.948  |  14.00
#> Overall            |        0.09 | 0.56 | [-1.01,  1.19] |  0.16 | 0.874  |
#> 
#> Uncertainty intervals (equal-tailed) and p-values (two-tailed) computed
#>   using a Wald z-distribution approximation.
sessionInfo()
#> R version 4.2.1 (2022-06-23)
#> Platform: x86_64-apple-darwin17.0 (64-bit)
#> Running under: macOS Catalina 10.15.7
#> 
#> Matrix products: default
#> BLAS:   /Library/Frameworks/R.framework/Versions/4.2/Resources/lib/libRblas.0.dylib
#> LAPACK: /Library/Frameworks/R.framework/Versions/4.2/Resources/lib/libRlapack.dylib
#> 
#> locale:
#> [1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8
#> 
#> attached base packages:
#> [1] stats     graphics  grDevices utils     datasets  methods   base     
#> 
#> other attached packages:
#> [1] parameters_0.18.2 emmeans_1.8.0     metafor_3.9-5     metadat_1.2-0    
#> [5] Matrix_1.4-1     
#> 
#> loaded via a namespace (and not attached):
#>  [1] rstudioapi_0.14    knitr_1.40         mathjaxr_1.6-0     magrittr_2.0.3    
#>  [5] insight_0.18.2     xtable_1.8-4       lattice_0.20-45    rlang_1.0.4       
#>  [9] fastmap_1.1.0      stringr_1.4.1      tools_4.2.1        grid_4.2.1        
#> [13] nlme_3.1-157       xfun_0.32          coda_0.19-4        cli_3.3.0         
#> [17] bayestestR_0.12.1  withr_2.5.0        htmltools_0.5.3    datawizard_0.5.1  
#> [21] yaml_2.3.5         digest_0.6.29      lifecycle_1.0.1    fs_1.5.2          
#> [25] glue_1.6.2         evaluate_0.16      rmarkdown_2.16     reprex_2.0.2      
#> [29] stringi_1.7.8      estimability_1.4.1 compiler_4.2.1     mvtnorm_1.1-3

Created on 2022-09-03 with reprex v2.0.2

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the reported metafor::rma() example and inspect the model_parameters(res) entry point, comparing its subgroup estimates with predict() and emmeans(). Done means predictions and 95% confidence intervals match across all alloc subgroups; the issue names no repository files or tests.

Written by the indexing model from the issue text.

Assessment

Tech stack
r
Domain
data
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.