Project-MONAI / Project-MONAI/MONAI
Feature Request: Evaluation of Semantic Segmentation Metrics on a per-component basis
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 8.7k
- Forks
- 1.6k
- Avg merge
- 5d 1h
- Merged PRs (30d)
- 20
Description
Is your feature request related to a problem? Please describe.
Current MONAI segmentation metrics aggregate scores globally. In multi-instance medical segmentation tasks this can mask clinically relevant failure modes, e.g. missing small lesions while achieving high Dice scores due to dominant large structures. There is no native support for connected component aware evaluation except Panoptic Quality that treats each ground truth object as an individual evaluation unit.
Describe the solution you'd like
Add connected component aware variants of existing MONAI semantic segmentation metrics. The proposed CC (Connected-Component-wise) Metrics operate as thin wrappers around existing MONAI metrics and:
- Decompose the ground truth foreground into connected components
- Assign each voxel to its nearest ground truth component via a Voronoi partition based on a single Euclidean distance/feature transform.
- Evaluate standard MONAI metrics inside each component specific region of interest.
- Supports aggregation modes (patient: mean over components per case & overall: flat aggregation over all components)
- Integrate seamlessly with existing MONAI metrics such as DiceMetric, HausdorffDistanceMetric, SurfaceDistanceMetric, and SurfaceDiceMetric.
The implementation may be binary segmentation only first but coule be extended to multi-class approaches.
Describe alternatives you've considered
- Instance segmentation metrics. These require different model outputs and are not applicable to semantic segmentation pipelines.
- Panoptic Quality offers a part of the solution but relies on a fixed and threshold dependant overlap-based matching and only consideres IoU as a metric.
Additional context
Reference implementation exists as a standalone library CC Metrics (https://github.com/alexanderjaus/CC-Metrics), published with peer reviewed validation.
The approach is described in “Every Component Counts: Rethinking the Measure of Success for Medical Semantic Segmentation in Multi Instance Segmentation Tasks”, AAAI 2025.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reviewing MONAI's existing DiceMetric, HausdorffDistanceMetric, SurfaceDistanceMetric, and SurfaceDiceMetric implementations, then compare the proposed behavior with the standalone CC-Metrics library. Clarify how connected-component regions, Voronoi assignment, binary inputs, and patient versus overall aggregation should integrate with each metric. Done means supported metric variants are integrated and validated for the requested aggregation modes.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- computer-vision, machine-learning
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 42/100