awslabs / awslabs/graphstorm

Evaluation metrics, e.g., accuracy and auc, can NOT be used at the same time

Open
#540 0 comments 0 reactions 1 assignee Claimed by @zhjwy9343 View on GitHub
0.5
Dominant language
Python
Stars
452
Forks
76
PR merge metrics
No merged PRs in 30d

Description

For a classification task, cannot use all metrics at the same time.

Examples:
Accuracy and auc_roc requires different prediction return values. Therefore they cannot be used in one training/evaluation epoch.

Reasons:
GS uses `return_proba` to control the types of prediction values, which means the predict function can NOT return both types of values.

Suggested solution:
- Option 1: modify predict function to return both;
- Option 2: based on the metrics given, run one or twice of the predict function.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.