Lightning-AI / Lightning-AI/torchmetrics
Intersection Over Union (IoU) should support Shape N x C_1 x C_2 x ... x C_n
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 2.5k
- Forks
- 526
- Avg merge
- 6d 11h
- Merged PRs (30d)
- 5
Description
## 🐛 Bug
I was writing a Pull Request for Sorted AP metric and I figured "let's use torchmetrics's IOU function!". Unfortunately, unlike the [JaccardIndex](https://lightning.ai/docs/torchmetrics/stable/classification/jaccard_index.html), torchmetric's [IntersectionOverUnion](https://lightning.ai/docs/torchmetrics/stable/detection/intersection_over_union.html) does not take multiple dimensions.
Infact, it only allows the very limiting shape of (N, 4). This is not an oversight; it's a major flaw hurting torchmetric's credibility because IoU is the most well-known computer vision metric. A user will first look for IntersectionOverUnion and NOT JaccardIndex and chances are that they'll be disappointed.
I propose that IntersectionOverUnion be a wrapper over [MultilabelJaccardIndex](https://lightning.ai/docs/torchmetrics/stable/classification/jaccard_index.html#torchmetrics.classification.MultilabelJaccardIndex)
I am willing to make a PR for it.
### To Reproduce
See any example from here. [IntersectionOverUnion](https://lightning.ai/docs/torchmetrics/stable/detection/intersection_over_union.html)
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the IntersectionOverUnion entry point and compare its documented input shape with the MultilabelJaccardIndex behavior linked in the issue. Verify the intended N x C_1 x C_2 x ... x C_n support against the existing IntersectionOverUnion examples, then add coverage showing the expanded dimensions work without breaking current usage.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- computer-vision, machine-learning
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 72/100