VGT training memory requirement

I am able to train it with one 2080Ti with 12 GB of VRAM but I would have to resort to having a batch size of 1.

Training with a batch size of 4 and 4 2080Tis throws out of memory after some time even with mixed precision enabled.

Please let me know about the:
1) Normal memory requirements for training
2) Any ways to reduce the memory required.

Thanks very much in advance.