TriviaQA LR scheduler code issue

Hi,

For single gpu training using the triviaqa code script, the learning rate goes to 0 in the first epoch itself.

Possible reasons: For a batchsize of 1, the global_step in pytorch_lightning increases with each batch of size 1 returned by the data_loader. It doesn't correspond to the number of optimizer steps. The LR scheduler was written with accumulated gradient batch size and thus the learning rate goes to 0 within the first epoch itself.

Thanks.
Apoorv