`CocoDetection()` doesn't work using some train and validation images with some annotations

🚀 The feature

CocoDetection() doesn't work using stuff_train2017_pixelmaps with stuff_train2017.json and using stuff_val2017_pixelmaps with stuff_val2017.json as shown below:

from torchvision.datasets import CocoDetection

pms_stf_train2017_data = CocoDetection(
    root="data/coco/anns/stuff_trainval2017/stuff_train2017_pixelmaps",
    annFile="data/coco/anns/stuff_trainval2017/stuff_train2017.json"
)

pms_stf_val2017_data = CocoDetection(
    root="data/coco/anns/stuff_trainval2017/stuff_val2017_pixelmaps",
    annFile="data/coco/anns/stuff_trainval2017/stuff_val2017.json"
)

pms_stf_train2017_data[0] # Error

pms_stf_val2017_data[0] # Error

FileNotFoundError: [Errno 2] No such file or directory: '/.../data/coco/anns/stuff_trainval2017/stuff_train2017_pixelmaps/000000000009.jpg'

FileNotFoundError: [Errno 2] No such file or directory: '/../data/coco/anns/stuff_trainval2017/stuff_val2017_pixelmaps/000000000139.jpg'

And, CocoDetection() doesn't work using panoptic_train2017 with panoptic_train2017.json and using panoptic_val2017 and panoptic_val2017.json as shown below:

from torchvision.datasets import CocoDetection

pan_train2017_data = CocoDetection(
    root="data/coco/anns/panoptic_trainval2017/panoptic_train2017",
    annFile="data/coco/anns/panoptic_trainval2017/panoptic_train2017.json"
) # Error

pan_val2017_data = CocoDetection(
    root="data/coco/anns/panoptic_trainval2017/panoptic_val2017",
    annFile="data/coco/anns/panoptic_trainval2017/panoptic_val2017.json"
) # Error

KeyError: 'id'

Motivation, pitch

So, CocoDetection() should support for them.

Alternatives

No response

Additional context

No response

Start at the torchvision.datasets.CocoDetection entry point and reproduce the provided stuff and panoptic dataset examples. Inspect how their image paths and annotation identifiers are interpreted, then verify that the listed train and validation datasets load and can retrieve their first item without the reported errors.