[BUG] _RuntimeError: p.is_cuda() INTERNAL ASSERT FAILED at "/home/runner/work/Uni-Core/Uni-Core/csrc/adam/interface.cpp":10, please report a bug to PyTorch. p must be a CUDA tensor

### Describe the bug

I want to fine-tune a pre-trained model using the BACE dataset.
However, when I run the fine-tuning code, I encounter the following error.
It seems that when the pre-trained model is loaded, the model parameters are in CPU format.
What should I do to achieve my goal?

File "/root/miniconda3/envs/py38-unimol/lib/python3.8/site-packages/torch/optim/optimizer.py", line 484, in wrapper
out = func(*args, **kwargs)
File "/root/miniconda3/envs/py38-unimol/lib/python3.8/site-packages/unicore/optim/fused_adam.py", line 130, in step
unicore_fused_adam.adam(p.data,
**RuntimeError: p.is_cuda() INTERNAL ASSERT FAILED at "/home/runner/work/Uni-Core/Uni-Core/csrc/adam/interface.cpp":10, please report a bug to PyTorch. p must be a CUDA tensor**
Traceback (most recent call last):
File "finetuning.py", line 101, in
main()
File "finetuning.py", line 98, in main
subprocess.run(cmd, check=True)
File "/root/miniconda3/envs/py38-unimol/lib/python3.8/subprocess.py", line 512, in run
raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['python', '/root/miniconda3/envs/py38-unimol/bin/unicore-train', '../example_data/', '--task-name', 'bace', '--user-dir', '../unimol', '--train-subset', 'train', '--valid-subset', 'valid', '--conf-size', '11', '--num-workers', '0', '--ddp-backend', 'c10d', '--dict-name', 'dict.txt', '--task', 'mol_finetune', '--loss', 'finetune_cross_entropy', '--arch', 'unimol_base', '--classification-head-name', 'bace', '--num-classes', '2', '--optimizer', 'adam', '--adam-betas', '(0.9, 0.99)', '--adam-eps', '1e-6', '--clip-norm', '1.0', '--lr-scheduler', 'polynomial_decay', '--lr', '0.0001', '--warmup-ratio', '0.06', '--max-epoch', '1', '--batch-size', '32', '--pooler-dropout', '0.1', '--update-freq', '1', '--seed', '0', '--fp16', '--fp16-init-scale', '4', '--fp16-scale-window', '256', '--log-interval', '100', '--log-format', 'simple', '--validate-interval', '1', '--keep-last-epochs', '10', '--finetune-from-model', '/root/PythonProject/Uni-Mol-main/Uni-Mol-main/unimol/notebooks/mol_pre_no_h_220816.pt', '--best-checkpoint-metric', 'valid_agg_auc', '--patience', '20', '--save-dir', './save_demo', '--only-polar', '0', '--maximize-best-checkpoint-metric', '--fp16', '--device-id', '0']' returned non-zero exit s

### Uni-Mol Version

Uni-Mol

### Expected behavior

What should I do to achieve my goal?

### To Reproduce

_No response_

### Environment

_No response_

### Additional Context

_No response_