ModelOriented / ModelOriented/DALEX

using yardstick (or other package) for performance calculations

Open
#495 0 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

R 🐳
Dominant language
Python
Stars
1.5k
Forks
172
PR merge metrics
No merged PRs in 30d

Description

The loss functions included are great but are somewhat limited.

There are a lot of R packages that could expand the types of loss functions (but at the loss of R:python parity).

If you were to use yardstick, for example, the benefits would be:

  • More metrics
  • Data on direction (e.g larger-is-better) for each metrics. You wouldn't have to do 1 - AUC anymore.
  • yardstick can compute multiple metrics at once.
  • Numerous multi-class metrics
  • Metrics for censored regression
  • Extensible for user-defined metrics

The downside to the current system is that you might optimize your model on a set of performance scores and judge feature importance on some other score.

Let use know if we can help or put in a PR.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Review the existing loss-function implementation and compare its coverage with yardstick's metrics, direction metadata, multi-metric support, multiclass metrics, censored regression, and user-defined metrics. The issue names no files, tests, entry points, or completion criteria, so confirm the intended scope and R/Python parity requirements with maintainers before coding.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, r
Domain
machine-learning
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.