Instance Segmentation Confusion Matrix

Search before asking

• I have searched the Supervision issues and found no similar feature requests.

Description

The current evaluation api supports confusion matrix only for object detection. Giving a view on how the model is performing for instance segmentation per class is desirable.

Slight change in API expected:

@dataclass
class ConfusionMatrix:
    matrix: np.ndarray
    segmentationMatrix: np.ndarray
    classes: List[str]
    conf_threshold: float
    iou_threshold: float

    @classmethod
    def from_detections(
        cls,
        predictions: List[sv.Detections],
        target: List[sv.Detections],
        classes: List[str],
        conf_threshold: float = 0.3,
        iou_threshold: float = 0.5
    ) -> ConfusionMatrix:
        pass

   @classmethod
    def benchmark(
        cls,
        dataset: sv.DetectionDataset,
        callback: Callable[[np.ndarray], sv.Detections],
        conf_threshold: float = 0.3,
        iou_threshold: float = 0.5
    ) -> ConfusionMatrix:
        pass

    def plot(self, target_path: str) -> None:
        pass

Discussion points:

1. Note that I'm not changing the previous name matrix to detectionMatrix for backward compatibility. If we believe it's not necessary and brings readability for the long term, can make that change.
2. Should this be an opt-in feature. Not sure what the user behaviour is. If the 90 percentile user wants both let's not. If it's the other way around, we can instead make a separate confusion matrix class called SegmentationConfusionMatrix

Use case

No response

Additional

No response

Are you willing to submit a PR?

• Yes I'd like to help by submitting a PR!

Start by locating the existing object-detection confusion-matrix evaluation API and its from_detections, benchmark, and plot entry points. Review the eight-comment discussion before choosing the API shape; done means instance-segmentation results can be evaluated per class and represented and plotted without breaking the existing matrix behavior.