RuntimeError: Only Tensors created explicitly by the user (graph leaves) support the deepcopy protocol at the moment

Nobody has claimed this yet.

### Bug description

This is a new issue (I've been using this version of pytorch-lightning and model for over a year) so I'm sure I've somehow introduced a breaking change. My problem is I can't figure out from the trace where it actually occurs. It seems to be originating from `stochastic_weight_avg.py` but the thing is I hit CTRL+C before it happens, if I don't it doesn't occur!

Prior to this error, I would hit CTRL+C during Training/evaluation, lightning would catch this, and then move on to testing the model. Now, instead of doing this, I get this runtime error.

If I don't hit CTRL+C then the runtime error does not occur it just finishes as expected and then moves on to testing.
Any help or insights are greatly appreciated!

### What version are you seeing the problem on?

_No response_

### How to reproduce the bug

```python
Nothing to see really, just boilerplate standard trainer invocation.
```

### Error messages and logs

```
rank_zero_warn("Detected KeyboardInterrupt, attempting graceful shutdown...")
Test Shape torch.Size([27157, 10, 5])
Traceback (most recent call last):
File "/home/tons/code/py/one-shot/scripts/autoencoding.py", line 310, in
train_wavetek()
File "/home/tons/code/py/one-shot/scripts/autoencoding.py", line 269, in train_wavetek
trainer.test(model, dataloaders=test_loader)
File "/home/tons/code/py/one-shot/venv/lib/python3.9/site-packages/pytorch_lightning/trainer/trainer.py", line 911, in test
return self._call_and_handle_interrupt(self._test_impl, model, dataloaders, ckpt_path, verbose, datamodule)
File "/home/tons/code/py/one-shot/venv/lib/python3.9/site-packages/pytorch_lightning/trainer/trainer.py", line 685, in _call_and_handle_interrupt
return trainer_fn(*args, **kwargs)
File "/home/tons/code/py/one-shot/venv/lib/python3.9/site-packages/pytorch_lightning/trainer/trainer.py", line 954, in _test_impl
results = self._run(model, ckpt_path=self.tested_ckpt_path)
File "/home/tons/code/py/one-shot/venv/lib/python3.9/site-packages/pytorch_lightning/trainer/trainer.py", line 1136, in _run
self.call_hook("on_before_accelerator_backend_setup")
File "/home/tons/code/py/one-shot/venv/lib/python3.9/site-packages/pytorch_lightning/trainer/trainer.py", line 1495, in call_hook
callback_fx(*args, **kwargs)
File "/home/tons/code/py/one-shot/venv/lib/python3.9/site-packages/pytorch_lightning/trainer/callback_hook.py", line 38, in on_before_accelerator_backend_setup
callback.on_before_accelerator_backend_setup(self, self.lightning_module)
File "/home/tons/code/py/one-shot/venv/lib/python3.9/site-packages/pytorch_lightning/callbacks/stochastic_weight_avg.py", line 142, in on_before_accelerator_backend_setup
self._average_model = deepcopy(pl_module)
File "/usr/local/lib/python3.9/copy.py", line 172, in deepcopy
y = _reconstruct(x, memo, *rv)
File "/usr/local/lib/python3.9/copy.py", line 270, in _reconstruct
state = deepcopy(state, memo)
File "/usr/local/lib/python3.9/copy.py", line 146, in deepcopy
y = copier(x, memo)
File "/usr/local/lib/python3.9/copy.py", line 230, in _deepcopy_dict
y[deepcopy(key, memo)] = deepcopy(value, memo)
File "/usr/local/lib/python3.9/copy.py", line 146, in deepcopy
y = copier(x, memo)
File "/usr/local/lib/python3.9/copy.py", line 230, in _deepcopy_dict
y[deepcopy(key, memo)] = deepcopy(value, memo)
File "/usr/local/lib/python3.9/copy.py", line 146, in deepcopy
y = copier(x, memo)
File "/usr/local/lib/python3.9/copy.py", line 205, in _deepcopy_list
append(deepcopy(a, memo))
File "/usr/local/lib/python3.9/copy.py", line 153, in deepcopy
y = copier(memo)
File "/home/tons/code/py/one-shot/venv/lib/python3.9/site-packages/torch/_tensor.py", line 89, in __deepcopy__
raise RuntimeError("Only Tensors created explicitly by the user "
RuntimeError: Only Tensors created explicitly by the user (graph leaves) support the deepcopy protocol at the moment

```

### Environment

Current environment

```
#- Lightning Component (e.g. Trainer, LightningModule, LightningApp, LightningWork, LightningFlow): Trainer
#- PyTorch Lightning Version): 1.5.10
#- Lightning App Version:
#- PyTorch Version: 1.12.1+cu116
#- Python version: 3.9.11
#- OS: Ubuntu 18.04
#- CUDA/cuDNN version: 11.7
#- GPU models and configuration: RTX3090Ti
#- How you installed Lightning(`conda`, `pip`, source): pip
#- Running environment of LightningApp (e.g. local, cloud): local
```

### More info

_No response_

cc @carmocca

Start with callbacks/stochastic_weight_avg.py at on_before_accelerator_backend_setup, then trace Trainer._call_and_handle_interrupt and the test path shown in the stack trace. Reproduce the failure by interrupting training before trainer.test, and compare that flow with an uninterrupted run. Done means the interrupt-to-test path no longer raises the reported deepcopy error, with regression coverage for the behavior.

Written by the indexing model from the issue text.