Reproducing Llama results from the paper

Hey there,

firstly, thanks for the nice work!

I am attempting to reproduce the results from the paper. I re-ran the experiments with 10 seeds (averaging the results). However, I am only reproducing the numbers for 5 of 7 of the tasks, which do not require an LLM judge.

My results are the following:

| | BioASQ | BioRED | DiscMT | EvInf | MultiCite | SciERC | SciFact | SciFact |
|------------------------|-----------------------------|-----------------------------|-----------------------------|---------------------------------|--------------------------------|-----------------------------|------------------------------|------------------------------|
| | F1 | F1 | BLEU | "Fuzzy" F1 | F1 | F1 | F1-Label | F1-Token |
| Llama-3-8B-Inst. (original) | 43.3 | 40.3 | 37.3 | 13.5 | 37.9 | 25.4 | 42.3 | 40.1 |
| **Llama-3-8B-Inst. (reproduced)** | 43.1 ± 1.5 | 40.1 ± 1.1 | 36.8 ± 2.6 | 15.2 ± 1.7 | 34.9 ± 8.2 | **13.2** ± 4.6 | **22.0** ± 7.7 | **20.6** ± 7.2 |

I am uncertain what the reason could be that the reproduced results for SciERC and SciFact are different compared to the original. Do you know what could be the cause of this?

There is a slight change in the `--model_args` due to memory issues. I added `gpu_memory_utilization` or `max_model_len` and removed `tensor_parallel_size`. I am running the following command:
```
python -m lm_eval \
--include_path ./sciriff/eval/eleuther_templates/general \
--model vllm \
--model_args pretrained=meta-llama/Meta-Llama-3-8B-Instruct,dtype=float16,gpu_memory_utilization=0.85,max_model_len=5120 \
--gen_kwargs max_gen_toks=1024 \
--tasks bioasq_list_qa,biored_ner,discomat_te,evidence_inference,multicite_intent_classification,scierc_ner,scifact_entailment \
--batch_size auto \
--output_path results/ \
--seed 42 \
--predict_only \
--log_samples
```

And I am using the following seeds: 42, 1337, 9876, 12345, 999999, 98765, 5555, 2024, 267, 10.

I am running experiments on a single RTX A6000 (48 GB) using CUDA 12.4, driver version 550.90.07, and Python 3.10.13 with the following versions of the packages:
```
huggingface-hub==0.24.5
jinja2==3.1.4
jsonschema==4.23.0
https://github.com/EleutherAI/lm-evaluation-harness.git@e74ec966556253fbe3d8ecba9de675c77c075bce
nltk==3.8.1
openai==1.37.2
pandas==2.2.2
pyyaml==6.0.1
rouge_score==0.1.2
spacy==3.7.5
vllm==0.5.4
```