apex.parallel.convert_syncbn_model has gradient overflow until the loss scale is reduced to zero

I am using multi GPU training. Previously I was using DDP from PyTorch with synchronized batch normalization. Now I'm trying to do the same using apex. The problem happens when I try to train the model using convert_syncbn_model. Basically, when I put this line, the loss scale gets re-adjusted until it becomes zero and the loss outputs NaNs. Bellow, I put a snippet of my code:

torch.cuda.set_device(gpu) net.cuda(gpu) net = apex.parallel.convert_syncbn_model(net) net, optimizer = apex.amp.initialize(net, optimizer, opt_level='O1') net = DDP(net)

with apex.amp.scale_loss(loss, optimizer) as scaled_loss: scaled_loss.backward()

Everything works fine when I do not use apex.parallel.convert_syncbn_model.

System:
Ubuntu: 16.04
Pytorch: 1.3.1 (the same happens with 1.4)
Apex: 0.1

Start with the reported training sequence using apex.parallel.convert_syncbn_model, apex.amp.initialize, and DDP, and compare it with the working sequence that omits synchronized batch normalization. Reproduce the loss-scale reductions and NaN loss on the listed Ubuntu, PyTorch, and Apex versions; done means identifying the cause and preventing the loss scale from reaching zero.