easystats / easystats/performance

check_model for GAMs

Open
#405 3 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

Enhancement :boom:
Dominant language
R
Stars
1.2k
Forks
109
Avg merge
6h 34m
Merged PRs (30d)
8

Description

Currently it's not supported, but some specific metrics could be implemented (e.g., for checking the number of knots k):

m1 <- mgcv::gam(Sepal.Length ~ s(Petal.Length, k = 3), data = iris)
m2 <- mgcv::gam(Sepal.Length ~ s(Petal.Length, k = 10), data = iris)

performance::check_model(m1)
#> Error: $ operator is invalid for atomic vectors

# Visual checks
mgcv::gam.check(m1)  # mgcv::qq.gam(m1)

#> 
#> Method: GCV   Optimizer: magic
#> Smoothing parameter selection converged after 4 iterations.
#> The RMS GCV score gradient at convergence was 6.533645e-07 .
#> The Hessian was positive definite.
#> Model rank =  3 / 3 
#> 
#> Basis dimension (k) checking results. Low p-value (k-index<1) may
#> indicate that k is too low, especially if edf is close to k'.
#> 
#>                   k'  edf k-index p-value  
#> s(Petal.Length) 2.00 1.97    0.87   0.045 *
#> ---
#> Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
    # The test is implemented in k.check
    # Low p-values may indicate that the basis dimension, k, has been set too low, especially if the reported edf is close to k', the maximum possible EDF for the term.
    mgcv::k.check(m1)
    #>                 k'      edf   k-index p-value
    #> s(Petal.Length)  2 1.973634 0.8679672  0.0375
    mgcv::k.check(m2)
    #>                 k'      edf   k-index p-value
    #> s(Petal.Length)  9 6.095448 0.9578142  0.2675

Created on 2022-03-18 by the reprex package (v2.0.1)

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start at performance::check_model(m1) and inspect how model checks are dispatched, then compare the mgcv::k.check(m1) and mgcv::k.check(m2) outputs. Done means GAM inputs no longer trigger the shown atomic-vector error and the relevant knot or basis-dimension diagnostics are reported for the examples.

Written by the indexing model from the issue text.

Assessment

Tech stack
r
Domain
data
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.