zero-shot evaluation latency

Unlike Wikitext perplexity as mentioned in the code, Evaluation times degrade significantly when using lm-eval. For example, in BoolQ dataset, when running on llamapretrained model from huggingface it takes about 1hour on rtx 3090 GPU. However for wrapped model as mentioned in your code, runtime takes more than 8 hours to complete evaluation. Can anyone answer my question?