Training vargfacenet model produces nan's

I started to train y2 model on retina dataset and it works fine (loss it not NaN) :
`CUDA_VISIBLE_DEVICES='0,1,2,3' python -u train.py --dataset retina --network y2 --loss arcface`

When i switch to vargfacenet network the loss get NaN value right away in the first batch :
`CUDA_VISIBLE_DEVICES='0,1,2,3' python -u train.py --dataset retina --network vargfacenet --loss arcface`
I thought this is because of the batch size so i tried other values for per-batch-size (64, 32) but it still fails.

Can someone explain that ?