Explore the effect of batchnorm

The batchnorm in the default retinanet model is a headache and a source of worry. We currently turn on batchnorm update statistics in validation step in main.py, but obviously don't allow the weights to update.

        #Get loss from "train" mode for batch norm statistics, but don't allow optimization
        self.model.train()
        with torch.no_grad():
            loss_dict = self.model.forward(images, targets)

This is awkward and looks questionable. Even for just appearances sake I'd like so hard evidence that batchnorm is a needed part of this. Relates to the whole 'batchnorm is evil magic' theme.

https://patrick-llgc.github.io/Learning-Deep-Learning/assets/papers/Devils_in_BatchNorm_yuxin_wu.pdf

Start with the validation step in main.py and the default RetinaNet model's batch normalization behavior. Run controlled experiments comparing validation with the current train-mode, no-gradient path against alternatives, using the linked batch normalization paper as background. Done means reporting hard evidence about whether batch normalization is needed and documenting the observed effect.