微调qwen2.5系列时报错
- Dominant language
- Python
- Stars
- 5.2k
- Forks
- 448
- Avg merge
- 3d 15h
- Merged PRs (30d)
- 26
Description
微调的是qwen2.5的模型,使用的配置模板均基于qwen1.5,进行qlora微调,会发生以下报错。
xtuner版本:`0.1.23`
报错内容:
```
model = self.train_loop.run() # type: ignore
File "/opt/conda/lib/python3.10/site-packages/mmengine/runner/loops.py", line 273, in run
self.runner.call_hook('before_train')
File "/opt/conda/lib/python3.10/site-packages/mmengine/runner/runner.py", line 1841, in call_hook
raise TypeError(f'{e} in {hook}') from None
TypeError: Qwen2RotaryEmbedding.forward() got an unexpected keyword argument 'seq_len' in
Traceback (most recent call last):
File "/workspace/xtuner/xtuner/tools/train.py", line 360, in
main()
File "/workspace/xtuner/xtuner/tools/train.py", line 356, in main
runner.train()
File "/opt/conda/lib/python3.10/site-packages/mmengine/runner/runner.py", line 1777, in train
model = self.train_loop.run() # type: ignore
File "/opt/conda/lib/python3.10/site-packages/mmengine/runner/loops.py", line 273, in run
self.runner.call_hook('before_train')
File "/opt/conda/lib/python3.10/site-packages/mmengine/runner/runner.py", line 1841, in call_hook
raise TypeError(f'{e} in {hook}') from None
TypeError: Qwen2RotaryEmbedding.forward() got an unexpected keyword argument 'seq_len' in
```
Contributor guide
Assessment
This issue has not been assessed yet.