Error occurs when editing Baichuan-13B

```
loss 3.28 = 3.28 + 0.0 avg prob of [Rishi Sunak] 0.0498
loss nan = nan + nan avg prob of [Rishi Sunak] nan
loss nan = nan + nan avg prob of [Rishi Sunak] nan
loss nan = nan + nan avg prob of [Rishi Sunak] nan
```

The gradient of delta weight becomes `nan` after the first backward operation.

By using:

```python
with torch.autograd.detect_anomaly():
loss.backward()
```

We caught a runtime error by the script.

```
RuntimeError: Function 'MmBackward0' returned nan values in its 0th output.
```

I suppose that it may be related to the alibi attention masks of Baichuan-13B.