Caltech-101 doesn't load bboxes but it should.

🐛 Describe the bug

Annotation files in Caltech-101 have two keys: 'box_coord' and 'obj_contour'. Caltech101 PyTorch class appends just the contours, explicitly skipping 'box_coord'. This would be ok if coordinates of obj_contour were relative to the entire image. Unfortunately they are relative to the bboxes and thus are useless without loading box_coord.

There are two ways to fix this - load box_coord or transform obj_contour coordinates to be relative to the image.

Versions

My env is irrelevant as this is a clear bug in code on main:

torchvision/datasets/caltech.py#L95C9-L110C9

        target: Any = []
        for t in self.target_type:
            if t == "category":
                target.append(self.y[index])
            elif t == "annotation":
                data = scipy.io.loadmat(
                    os.path.join(
                        self.root,
                        "Annotations",
                        self.annotation_categories[self.y[index]],
                        f"annotation_{self.index[index]:04d}.mat",
                    )
                )
                target.append(data["obj_contour"])
        target = tuple(target) if len(target) > 1 else target[0]

cc @pmeier