cannot replicate DPO results of zephyr

I cannot replicate the DPO results for zephyr.

I use a modified version of config_full.yaml with the only difference being that I set gradient_accumulation_steps: 4 instead of 2 because I use 4 GPUs. I'm using all the correct versions of software as in setup.py. I resumed twice during training and its something that is inevitable with our cluster, but if resuming set seeds properly, this should not be a problem.

Code:
`ACCELERATE_LOG_LEVEL=info accelerate launch --config_file recipes/accelerate_configs/deepspeed_zero3.yaml --num_processes=4 scripts/run_dpo.py recipes/zephyr-7b-beta/dpo/config_full4.yaml

The results is here: https://huggingface.co/AlexiaJM/zephyr-7b-dpo-full-repnew. As you can see the numbers are slightly off from https://huggingface.co/alignment-handbook/zephyr-7b-dpo-full but not significantly.

These are the results from the MT-Bench:

########## First turn ##########
score
model turn
zephyr-7b-dpo-full 1 7.81250
zephyr-7b-dpo-full-repnew 1 7.5375

########## Second turn ##########
score
model turn
zephyr-7b-dpo-full 2 7.322785
zephyr-7b-dpo-full-repnew 2 7.125

########## Average ##########
score
model
zephyr-7b-dpo-full 7.569182
zephyr-7b-dpo-full-repnew 7.33125