Lightning-AI / Lightning-AI/torchmetrics

mAP Metric for 3D Bounding Boxes

Open
#3,398 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement
Dominant language
Python
Stars
2.5k
Forks
526
Avg merge
6d 11h
Merged PRs (30d)
5

Description

## 🚀 Feature

I would like to be able to compute mean-average-precision for 3D Bounding boxes to also be able to evaluate detection systems that are capable of processing 3D data, like point-clouds, voxels or other.

### Motivation

Currently whenever I want to compute the mAP with `torchmetrics.detection.mean_ap.MeanAveragePrecision` I am restricted to 2D data, meaning the parameter `num_boxes` is restricted to shape `(num_boxes, 4)`.

### Pitch

I would recommend to add a separate mAP class that only covers 3D bounding boxes. So to separate between
3D mAP and 2D mAP computation. The reason is that in 3D we have different algorithms to compute IoU and average precision etc.

### Alternatives

- Either extend the already existent `torchmetrics.detection.mean_ap.MeanAveragePrecision`.
- Or implement a complete new metric class.

### Additional context

Read more about mAP here:
- https://medium.com/@rohitbnsd12/3d-object-detection-evaluation-a88854534aac

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start at torchmetrics.detection.mean_ap.MeanAveragePrecision and review its current 2D box input contract. Compare the proposed separate class with extending the existing metric, then establish the 3D IoU and average-precision behavior and tests needed for point-cloud or voxel detection. Done means a documented, tested 3D mAP API.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, pytorch
Domain
computer-vision, machine-learning
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.