Is perplexity correctly computed?

Hello. I'm struggling with replicating the [reported perplexity](https://github.com/ggerganov/llama.cpp/discussions/2352) (~6) for LLaMa-2-7b.
I am using this simple code snippet:
```
import evaluate
import datasets

perplexity = evaluate.load("perplexity", module_type="metric")
input_texts = datasets.load_dataset("wikitext",
"wikitext-2-raw-v1",
split="test")["text"]
input_texts = [s for s in input_texts if s!='']
results = perplexity.compute(
model_id="sharpbai/Llama-2-7b-hf",
batch_size=4,
predictions=input_texts
)
print(results)
```
And I get among the results: `'mean_perplexity': 60.9764459149642`.
In [this tutorial](https://huggingface.co/docs/transformers/perplexity) it is computed "approximately" by flattening the dataset into a string and by computing the avg. sliding window perplexity. I still get a high perplexity.
I tried to change the model in the code snippet to `openai-community/gpt2` and the perplexity is above 600!
Does this depend on using the correct model class?
Thank you for any suggestion.

**EDIT:** I'm using the following versions
```
transformers 4.38.2
evaluate 0.4.1
datasets 2.18.0

```