Lightning-AI / Lightning-AI/torchmetrics

Add a new metric: Localization Recall Precision (LRP)

Open
#3,305 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement
Dominant language
Python
Stars
2.5k
Forks
526
Avg merge
6d 11h
Merged PRs (30d)
5

Description

## 🚀 Feature

Add a new metric: Localisation Recall Precision (LRP)

This metric provides a unified evaluation of object detection performance by jointly considering localisation error, recall, and precision. It is particularly useful as an alternative to mAP for assessing object detectors.

### Motivation

In my current and past projects, I have relied heavily on torchmetrics for evaluating deep learning models, especially for object detection tasks. While the library provides common metrics such as mAP, I also use Localisation Recall Precision (LRP), which provides a more interpretable and fine-grained assessment of detection quality.

LRP measures performance across false positives, false negatives, and localisation error simultaneously. I think it's a nice metric and including it in torchmetrics would make it more accessible for users working with object detection models.

### Pitch

**I would like to contribute by implementing Localisation Recall Precision** as a new metric within torchmetrics.

The initial implementation would:

Support standard object detection evaluation input formats (predictions and targets with bounding boxes, scores, and labels).

Compute LRP error.

Follow the existing structure and design of object detection metrics in the repo (similar to mAP).

**I would be happy to open a PR with an initial implementation and tests once the proposal is approved.**

### Alternatives

You could compute LRP outside of torchmetrics using standalone scripts. But this is less cool.

Relying solely on mAP.

### Additional context

The metric was first introduced in “Localisation Recall Precision (LRP): A New Performance Metric for Object Detection” (https://arxiv.org/pdf/1807.01696), and has since been used as an alternative evaluation metric in several object detection benchmarks and research papers.

I believe including this would expand the usefulness of torchmetrics for users working on detection tasks.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Review the existing object detection metrics, especially mAP, and read the cited LRP paper first. Define the supported prediction and target inputs, the expected LRP behavior, and the tests needed to show the new metric is complete.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, pytorch
Domain
machine-learning
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.