Out of memory XLM-roberta-large

Thanks for releasing your code. I replaced GPT2 with XLM-roberta-large, tensor dataset with LazyLoader, and did padding inside the data loader. Unfortunately, even with batch_size of 1, I got Cuda out of memory. Do you have any suggestions on how can I decrease memory usage? I have no problem with xlm-roberta-base.

The report concerns replacing GPT2 with XLM-roberta-large while using LazyLoader and padding inside the data loader; start by reproducing the CUDA out-of-memory failure with batch_size 1 and compare it with xlm-roberta-base. Done is not defined because the issue asks for suggestions rather than a specific change.

Written by the indexing model from the issue text.