easystats / easystats/parameters

`select_parameters()` outdated with respect to projpred

Open
#688 4 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Bug :bug: Low priority :sleeping:
Dominant language
R
Stars
499
Forks
45
Avg merge
3d 1h
Merged PRs (30d)
3

Description

It seems like select_parameters.stanreg() is outdated with respect to projpred:

  1. Line https://github.com/easystats/parameters/blob/ddf64b9bb0ffa91298641d30355d2b51940dcd36/R/select_parameters.stanreg.R#L25 uses element beta which doesn't exist in newer projpred versions anymore. I don't know when beta was removed exactly, but it doesn't exist in the current CRAN version 2.0.2 (from October 2020) anymore. To access the projected parameter draws in projpred, the as.matrix.projection() method can be used.
  2. Line https://github.com/easystats/parameters/blob/ddf64b9bb0ffa91298641d30355d2b51940dcd36/R/select_parameters.stanreg.R#L24 uses argument nv which doesn't exist in newer projpred versions anymore (at least in the current CRAN version 2.0.2 from October 2020, it doesn't exist anymore). This argument was replaced by a new argument called nterms.

Apart from that, I noticed that parameters doesn't make use of the projected parameter draws: Lines https://github.com/easystats/parameters/blob/ddf64b9bb0ffa91298641d30355d2b51940dcd36/R/select_parameters.stanreg.R#L25-L32 show that only the names of the predictors from the selected submodel are used, not their projected draws. I don't know if this was done on purpose or if it was even necessary to make projpred-based selections fit into the parameters workflow, but in principle, it would be preferable to use the projected parameter draws instead of refitting the model with only the selected predictors. Refitting the model with only the selected predictors throws away uncertainty inherent to the reference model, thereby leading to improper post-selection inference. Using the projected parameter draws after selection is even one of the key steps of the projpred workflow, I would say.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in R/select_parameters.stanreg.R at the referenced lines and compare its projpred calls with projpred 2.0.2, including as.matrix.projection(). Confirm the updated nv/nterms and beta/projected-draw behavior, and resolve whether selection should use projected draws rather than refitting; done means the function is compatible with the current API and its selection behavior is verified.

Written by the indexing model from the issue text.

Assessment

Tech stack
r
Domain
data
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.