Anchors not being properly distributed across feature maps?

*Edit: based on the rpn.py documentation, it sounds like this may be intended behavior for RPN - in which case this is really more of an issue in faster_rcnn, in the way that its default anchor sizes are specified. RPN docs note that "sizes[i] and aspect_ratios[i] can have an arbitrary number of elements," but this doesn't seem to be totally true - it looks like sizes[i] must have the same number of elements for all feature maps.

https://github.com/pytorch/vision/blob/d6ee8757eca7b74b98e5f0d434a565eb7b1c410b/torchvision/models/detection/rpn.py#L122

This zip statement will result in anchors being split up across each feature map, as opposed to computing all anchors at each feature map. For example, if anchor sizes are defined as ((8,), (16,), (32,), (64,), (128,)), the first feature map will only be processed using anchors size 8, the second feature map size 16, etc. Specifying a number of anchor sizes that is different from the number of feature maps will result in a crash, since the number of anchors and output predictions will not match.

Start with the RPN documentation and the zip statement in torchvision/models/detection/rpn.py at the referenced line. Verify how anchor sizes and aspect ratios are distributed across feature maps, including the default Faster R-CNN configuration and mismatched lengths. Done means the intended behavior is decided and the implementation and documentation agree without anchor or prediction-count mismatches.