pytorch / pytorch/ignite

Make optional argmax for y_pred in Confusion Matrix, Precision, Recall, Accuracy

Open
#822 9 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement needs-discussion
Dominant language
Python
Stars
4.8k
Forks
726
Avg merge
5d 21h
Merged PRs (30d)
5

Description

🚀 Feature

Today, the conditions on the input of the Confusion Matrix, (and Precision, Recall, Accuracy in multiclass case) are the following:

    - `y_pred` must contain logits and has the following shape (batch_size, num_categories, ...)
    - `y` should have the following shape (batch_size, ...) and contains ground-truth class indices
        with or without the background class. During the computation, argmax of `y_pred` is taken to determine predicted classes.

Taking argmax on y_pred can be an option if we would like to determine winning class by some other rule. Let's keep argmax as default behaviour if y_pred is (N, C, ...) and do not apply it if y_pred.shape == y.shape and (N, ...).

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by locating the Confusion Matrix, Precision, Recall, and Accuracy metric implementations and their tests. Check how predictions are handled for (N, C, ...) versus y_pred.shape == y.shape, then verify that default argmax behavior remains unchanged and same-shaped inputs support an alternative winning-class rule.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, pytorch
Domain
machine-learning
Issue type
Feature
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.