"RuntimeError: The size of tensor a (0) must match the size of tensor b (4096) at non-singleton dimension 1" (DPO + LoRA)

So I'm attempting to run the DPO LoRA script and I'm getting this error:
```
RuntimeError: The size of tensor a (0) must match the size of tensor b (4096) at non-singleton dimension 1
```

... when the `model.merge_and_load()` runs here:
```
base_model = AutoModelForCausalLM.from_pretrained(peft_config.base_model_name_or_path, **model_kwargs)
model = PeftModel.from_pretrained(base_model, model_args.model_name_or_path, revision=model_args.model_revision)
model.eval()
model = model.merge_and_unload()
```

Any ideas?