Issue with reproducing the paper results

Hello,

Could you please provide the evaluation metrics codes to produce Tables 1 and 2 ? After all my attempts I could not reproduce those numbers (I did get the main trend but the numbers are far off good as reported in the paper). Even when using the experiments.jsonl provided in the repository (and the one obtained from running scripts/run_experiments.py) I still can not get the numbers stated in the paper. Also, the both files contain 2,712 ckpts, not 2802 as mentioned on the paper.

Any help would be greatly appreciated. Thanks in advance

CC @valentinhofmann @davidheineman