Project-MONAI / Project-MONAI/MONAI
Create a confusion matrix based on tumors not per pixel for segmentation models
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 8.7k
- Forks
- 1.6k
- Avg merge
- 5d 1h
- Merged PRs (30d)
- 20
Description
Is your feature request related to a problem? Please describe.
Confusion matrix as true positive (TP), false positive (FP), true negative (TN) and false negative (FN) is a powerful accuracy metric that we have for classification.
We also have it segmentation tasks based on pixel counts. However, this is not meaningful for radiologist or data scientist since they care about number of tumors missed not number of pixel missed.
Describe the solution you'd like
It would be great if we have an accuracy metric that would do: for each label calculate connected component and compare it with the ground truth. If there is more than x % overlap then it is calculated as TP. if there is no ground truth then it is FP. Then we should check on the ground truth that doesn't have any inference should be counted as FN
We can ignore TN as it is about the background.
Describe alternatives you've considered
Manually have radiologist count the tumors and write values in excel files. This doesn't scale at all
Additional context
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by locating MONAI's existing segmentation confusion-matrix metrics and any connected-component utilities. Read how current metrics define true and false positives, then determine how tumor-level overlap and unmatched ground-truth components should be represented. Done means a documented metric counts TP, FP, and FN per tumor while excluding TN.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- computer-vision, machine-learning
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100