easystats / easystats/performance

`check_model` for `rms:lrm`

Open
#471 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Bug :bug: Enhancement :boom:
Dominant language
R
Stars
1.2k
Forks
109
Avg merge
6h 34m
Merged PRs (30d)
8

Description

There is some minimal support for rms::lrm already, for example:

model = rms::lrm(formula=am ~ mpg, data=mtcars, x=T, y=T, model=T)
print(performance::model_performance(model))

returns

Can't calculate log-loss.
Indices of model performance

R2 RMSE Sigma
0.466 0.392 0.995

This is rudimentary compared to the base logistic regression support:

model_base = stats::glm(formula=am ~ mpg, data=mtcars, family='binomial')
print(performance::model_performance(model_base))

Indices of model performance

AIC BIC Tjur's R2 RMSE Sigma Log_loss Score_log Score_spherical PCP
33.675 36.607 0.368 0.392 0.995 0.464 -9.303 0.093 0.695

And check_model(model) does not work at all:

Error in if (model$rank == 0) { : argument is of length zero
     ▆
  1. ├─base::withVisible(...)
  2. ├─performance::check_model(model)
  3. ├─performance:::check_model.default(model)
  4. │ ├─base::suppressWarnings(.check_assumptions_glm(x, minfo))
  5. │ │ └─base::withCallingHandlers(...)
  6. │ └─performance:::.check_assumptions_glm(x, minfo)
  7. │   └─performance:::.diag_qq(model)
  8. │     ├─base::sort(stats::rstandard(model, type = "pearson"), na.last = NA)
  9. │     ├─stats::rstandard(model, type = "pearson")
 10. │     └─stats:::rstandard.glm(model, type = "pearson")
 11. │       ├─stats::influence(model, do.coef = FALSE)
 12. │       └─stats:::influence.glm(model, do.coef = FALSE)
 13. │         └─stats::lm.influence(model, do.coef = do.coef, ...)
 14. └─global `<fn>`()

Error in if (model$rank == 0) { : argument is of length zero

So my questions is: do you plan to support rms::lrm, or is it not worth it since most common use cases can be handled with stats::glm?

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with check_model(model) and the check_model.default path shown in the traceback, especially .check_assumptions_glm and .diag_qq. Compare the existing rms::lrm behavior with stats::glm and model_performance(model), then verify that check_model works and the missing performance measures are supported without the reported error.

Written by the indexing model from the issue text.

Assessment

Tech stack
r
Domain
machine-learning
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.