dotnet / dotnet/machinelearning

Are we happy with Anomaly Detection metrics?

Open
#2,725 3 comments 0 reactions 0 assignees View on GitHub
area-Core enhancement Priority:2
Dominant language
C#
Stars
9.4k
Forks
2k
Avg merge
2d 20h
Merged PRs (30d)
11

Description

Right now it has two properties.
AUC and DetectionRateAtKFalsePosititives.
We describe `DetectionRateAtKFalsePosititives`
as:
```
/// This is computed as follows:
/// 1.Sort the test examples by the output of the anomaly detector in descending order of scores.
/// 2.Among the top K False Positives, compute ratio : (True Positive @ K) / (Total anomalies in test data)
/// Example confusion matrix for anomaly detection:
/// Anomalies (in test data) | Non-Anomalies (in test data)
/// Predicted Anomalies : TP | FP
/// Predicted Non-Anomalies : FN | TN
```
and we expose nothing of that.
No True positive, no total anomalies, no K which user need to save somewhere else.

Should we expand this metrics right now or wait till v1.0 release?

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.