List of metrics in R and elsewhere, and which ones are implemented in MLJ
Nobody has claimed this yet.
- Dominant language
- Julia
- Stars
- 163
- Forks
- 46
- Avg merge
- 1d 18h
- Merged PRs (30d)
- 5
Description
https://mlr.mlr-org.com/articles/tutorial/measures.html
binary classification
| MLR | MLJBase | Comment |
|---|---|---|
| acc | accuracy |
✔️ |
| auc | auc |
✔️ |
| bac | bac, bacc, balanced_accuracy |
✔️ ; we use the sklearn definition which is also valid for multiclass |
| ber | missing | |
| brier | BrierScore |
maybe worth adding a shortcut? |
| brier scaled | missing? | maybe worth checking? |
| f1 | f1, f1score |
✔️ |
| fdr | fdr, falsediscovery_rate |
✔️ |
| fn | fn, falsenegative |
✔️ |
| fnr | fnr, falsenegative_rate, miss_rate |
✔️ |
| fp | fp, falsepositive |
✔️ |
| fpr | fpr, falsepositive_rate, fallout |
✔️ |
| gmean | missing | |
| gpr | missing | |
| kappa | missing | |
| logloss | cross entropy ? | check |
| lsr | missing | |
| mcc | mcc, mathews_correlation |
✔️ |
| mmce | missing | |
| multiclass au1p | missing | 👀 |
| multiclass au1u | missing | |
| multiclass aunp | missing | |
| mutliclass aunu | missing | |
| multiclass brier | missing | |
| npv | npv |
✔️ |
| ppv | ppv, precision |
✔️ |
| qsr | missing | |
| ssr | missing | |
| tn | truenegative, tn |
✔️ |
| tnr | truenegative_rate, tnr, specificity, selectivity |
✔️ |
| tp | truepositive, tp |
✔️ |
| tpr | truepositive_rate, tpr, recall, sensitivity, hit_rate |
✔️ |
| wkappa | missing |
multiclass classification
| MLR | MLJBase | Comment |
|---|---|---|
| acc | accuracy | |
| f1 | ✔️ | |
| hamloss | missing | |
| ppv | ✔️ | |
| subset01 | missing | |
| tpr | ✔️ |
regression
some of these may be available in LossFunctions (?) + I did this one on the top of my head so may be worth double checking
| MLR | MLJBase | Comment |
|---|---|---|
| arsq | missing | |
| expvar | missing | |
| kendalltau | missing | |
| mae | mav | check |
| mape | missing | |
| medae | missing | |
| medse | missing | |
| mse | mse | |
| msle | missing | |
| rae | missing | |
| rmse | rms | |
| rmsle | rmsl | |
| rrse | missing | |
| rsq | missing | |
| sae | missing | |
| spearmanrho | missing | |
| sse | missing |
survival analysis
we don't have that yet
cluster analysis
we don't have any of those but probably should
Cost-senstive classification
we don't have that
General performance model
(?)
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by auditing the listed metrics against MLJBase and checking whether some regression measures are available through LossFunctions. Resolve the entries marked missing, uncertain, or needing verification and define which metric families are in scope. Done means the supported and unsupported measures are accurately documented and any additions have corresponding coverage.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- julia
- Domain
- machine-learning
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100