Is it possible to finetune the pretrained model on casual language modeling or text summarization?

Hi,
Thanks for providing and presenting this nice work.

As mentioned in your paper, your attention pattern for modeling long sequences can be plugged into any pretrained transformer model.
I wonder if this repo covers code to finetune a pretrained LM (e.g. gpt-2) or your own released pretrained model on a new dataset for language modeling task?

If so, is it possible through PyTorch implementation or the CUDA kernel?
I would appreciate if you can guide me in this respect.