pytorch / pytorch/ignite

[Feature] Implement TopK wrapper class for metrics

Open
#3,568 13 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
4.8k
Forks
726
Avg merge
5d 21h
Merged PRs (30d)
5

Description

🚀 Feature

(Follow up to #3530 and #2631)

After getting in the HitRate@top_k metric for recommendation systems, it makes sense to have Recall@top_k as well. To implement this metric, as discussed we can add a top_k argument to existing Recall metric instead of creating a new class:

The idea is to use the _prepare_output method of _BasePrecision_Recall to convert our data to the required format that can then be used directly with same logic of if self._average == "samples": of Recall class:
for the true_positives, fetch the top-k indices of y_pred and gather those elements from y.
y can then also be directly be passed as actual_positives.
https://github.com/pytorch/ignite/blob/24535a0bb078398d1374657ce6f8c1f583cd467a/ignite/metrics/precision.py#L60

_, top_indices = torch.topk(y_pred, k=self._top_k, dim=1)
true_positives = torch.gather(y, 1, top_k_indices)

then in https://github.com/pytorch/ignite/blob/24535a0bb078398d1374657ce6f8c1f583cd467a/ignite/metrics/recall.py#L229
we can use this same logic for final metric calculation

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reading ignite/metrics/precision.py, especially _BasePrecision_Recall._prepare_output, and ignite/metrics/recall.py around the Recall calculation. Compare the requested top-k behavior with the existing Recall logic and the linked HitRate work; done means Recall supports the requested top-k calculation consistently with the stated approach.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, pytorch
Domain
machine-learning
Issue type
Feature
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.