stan-dev / stan-dev/projpred

Inconsistency in reference model formula between GAMs and GAMMs

Open
#384 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

additive
Dominant language
R
Stars
114
Forks
31
PR merge metrics
No merged PRs in 30d

Description

There is an inconsistency in the reference model formula between GAMs and GAMMs (here shown with objects from the unit tests):

> refmods$rstanarm.gam.gauss.stdformul.with_wobs.without_offs.trad$formula
y_gam_gauss ~ xco.1 + xco.2 + xco.3 + xca.1 + xca.2 + s(s.1)
> refmods$rstanarm.gamm.gauss.stdformul.with_wobs.without_offs.trad$formula
y_gamm_gauss ~ xco.1 + xco.2 + xco.3 + xca.1 + xca.2 + s.1 + 
    s(s.1) + (xco.1 | z.1)

so the GAM lacks the extra s.1 term. This is probably related to

> formula.gamm4(fits$rstanarm.gam.gauss.stdformul.with_wobs.without_offs)
y_gam_gauss ~ xco.1 + xco.2 + xco.3 + xca.1 + xca.2 + s(s.1)
> formula.gamm4(fits$rstanarm.gamm.gauss.stdformul.with_wobs.without_offs)
y_gamm_gauss ~ xco.1 + xco.2 + xco.3 + xca.1 + xca.2 + s.1 + 
    s(s.1) + (xco.1 | z.1)

However, this doesn't seem to have any consequences for the candidate models in the forward search: First running

debug(search_forward)
vs_gam <- varsel(refmods$rstanarm.gam.gauss.stdformul.with_wobs.without_offs.trad,
                 nclusters = 1, nclusters_pred = 1)

and then debugging search_forward() until size 2 gives

> cands
[1] "s(s.1)" "xco.1"  "xco.2"  "xco.3"  "xca.1"  "xca.2"  "s.1"

so both s.1 and s(s.1) are considered as candidates (as desired). Similarly, for the GAMM, we get from

vs_gamm <- varsel(refmods$rstanarm.gamm.gauss.stdformul.with_wobs.without_offs.trad,
                  nclusters = 1, nclusters_pred = 1)

and debugging search_forward() until size 2:

> cands
[1] "s(s.1)"    "(1 | z.1)" "xco.1"     "xco.2"     "xco.3"     "xca.1"     "xca.2"     "s.1"      

so again, both s.1 and s(s.1) are considered as candidates (as desired).

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Reproduce the discrepancy using the referenced unit-test objects and compare the GAM and GAMM formulas returned by formula.gamm4. Trace how varsel() and search_forward() construct reference models and candidate terms, then verify that the formulas are consistent without losing either s.1 or s(s.1) from the candidate list.

Written by the indexing model from the issue text.

Assessment

Tech stack
r
Domain
machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.