Lightning-AI / Lightning-AI/torchmetrics

Intersection Over Union (IoU) should support Shape N x C_1 x C_2 x ... x C_n

Open
#3,262 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug / fix help wanted
Dominant language
Python
Stars
2.5k
Forks
526
Avg merge
6d 11h
Merged PRs (30d)
5

Description

## 🐛 Bug

I was writing a Pull Request for Sorted AP metric and I figured "let's use torchmetrics's IOU function!". Unfortunately, unlike the [JaccardIndex](https://lightning.ai/docs/torchmetrics/stable/classification/jaccard_index.html), torchmetric's [IntersectionOverUnion](https://lightning.ai/docs/torchmetrics/stable/detection/intersection_over_union.html) does not take multiple dimensions.

Infact, it only allows the very limiting shape of (N, 4). This is not an oversight; it's a major flaw hurting torchmetric's credibility because IoU is the most well-known computer vision metric. A user will first look for IntersectionOverUnion and NOT JaccardIndex and chances are that they'll be disappointed.

I propose that IntersectionOverUnion be a wrapper over [MultilabelJaccardIndex](https://lightning.ai/docs/torchmetrics/stable/classification/jaccard_index.html#torchmetrics.classification.MultilabelJaccardIndex)

I am willing to make a PR for it.

### To Reproduce

See any example from here. [IntersectionOverUnion](https://lightning.ai/docs/torchmetrics/stable/detection/intersection_over_union.html)

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the IntersectionOverUnion entry point and compare its documented input shape with the MultilabelJaccardIndex behavior linked in the issue. Verify the intended N x C_1 x C_2 x ... x C_n support against the existing IntersectionOverUnion examples, then add coverage showing the expanded dimensions work without breaking current usage.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, pytorch
Domain
computer-vision, machine-learning
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
72/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.