Improve testing for models

We currently have very limited testing for the models in torchvision.

In most cases, we only test for the output shape to be correct, see
https://github.com/pytorch/vision/blob/8635be94d1216f10fb8302da89233bd86445e449/test/test_models.py#L29-L59
While this allows to catch bugs such as Py2-Py3 differences, it is not enough to ensure that refactorings of the models keep the behavior the same (and that pre-trained weights are still valid).

We should come up with a better testing strategy.

Ideally, we would not want to download the pre-trained weights, because this can make the tests prone to failure due to IO issues.

One possible approach would be to fix the random seed and initialize the models in a particular way and compare the output we get for a particular input with an expected output, ensuring that the output is non-trivial (such as all-zeros or empty, which could happen due to ReLU or for detection models). Something in the lines of

torch.manual_seed(42)
m = torchvision.models.resnet18(num_classes=2)
torch.manual_seed(42)
i = torch.rand(2, 3, 224, 224)
output = m(i)
expected_output = torch.tensor([[0.0002, 0.3], [0.245, 0.001]])
assert output.equals(expected_output)

The problem with this approach is that we do not guarantee that the RNG is the same between different versions of PyTorch, which means that this way of testing the model would lead to failures if we change PyTorch's RNG.

This issue will be particularly important to be addressed soon, because we will be adding some slight modifications to the model implementations in order for them to be traceable / ONNX-exportable. @lara-hdr is currently looking into making the torchvision models ONNX-ready.

cc @fbbradheintz , who showed interest in addressing this issue.