facebookresearch / facebookresearch/detectron2

How to "visualise" the images which go inside the model for training? (looking at input to model)

Open
#4,675 0 comments 1 reaction 0 assignees View on GitHub
enhancement
Dominant language
Python
Stars
34.7k
Forks
7.9k
PR merge metrics
No merged PRs in 30d

Description

How can I see my inputs to `Detectron 2: Faster RCNN` Object detection model? I need to debug my model to see both the `X` (Image) and `Y` (bounding Box coordinates, classes, labels etc). I am using some custom augmentations and [in this github comment](https://github.com/facebookresearch/detectron2/issues/791#issuecomment-581223086) it says
>You can loop over the data loader with `for data in data_loader` and visualize them

But how do I do that? I can easily run a loop through my custom augmentations but what I want to know is that **how do I see the inputs just before going to inputs**?

I am using custom augmentations with `mapper` like:

```
def custom_mapper(dataset_dict, transform_list = None):

if transform_list is None:
transform_list = [T.RandomBrightness(0.8, 1.2),
T.RandomContrast(0.8, 1.2),
T.RandomSaturation(0.8, 1.2),
]

dataset_dict = copy.deepcopy(dataset_dict)
image = utils.read_image(dataset_dict["file_name"], format="BGR")

image, transforms = T.apply_transform_gens(transform_list, image)
tensor_image = torch.as_tensor(image.transpose(2, 0, 1).astype("float32"))
dataset_dict["image"] = tensor_image

annos = [
utils.transform_instance_annotations(obj, transforms, image.shape[:2])
for obj in dataset_dict.pop("annotations")
if obj.get("iscrowd", 0) == 0
]
instances = utils.annotations_to_instances(annos, image.shape[:2])
dataset_dict["instances"] = utils.filter_empty_instances(instances)

# visualizer = Visualizer(dataset_dict["image"].numpy().transpose(1,2,0).astype(np.uint8)[:, :, ::-1], scale=0.5)
# out = visualizer.draw_dataset_dict(dataset_dict)
# cv2.imwrite(str(np.random.rand())+".jpg", out.get_image()[:, :, ::-1], )

return dataset_dict

class AugTrainer(DefaultTrainer): # Trainer with augmentations
@classmethod
def build_train_loader(cls, cfg):
return build_detection_train_loader(cfg, mapper=custom_mapper)
```

I have tried using these 3 commented out lines (just before `return dataset_dict`) to save the image so that I can see the what do the images look like but how can I know whether they form the proper BB, Class and more importantly, these are the exact images which go inside the model and there isn't something else which is altering my input?

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.