Memory needed for evaluation of TriviaQA

Hello,
I was wondering how much GPU memory was needed to evaluate on TriviaQA?

From the cheatsheet scripts and the evaluation code, I understand that you used the first 4096 tokens for evaluation with with batch size 1. Is this correct?
I'm asking because I'm trying to reproduce the leaderboard results and I keep getting OOM when trying to evaluate for more than the first 1024 tokens.

Thanks!