transformers-4.45版本以上Qwen2RotaryEmbedding.forward接口报错

Open

#978 opened on Dec 26, 2024 1 comment 0 reactions 0 assignees View on GitHub

Dominant language: Python
Stars: 5.2k
Forks: 448
Avg merge: 3d 15h
Merged PRs (30d): 26

Description

https://github.com/InternLM/xtuner/blob/90192ffe42612b0f88409432e7b4860294432bcc/xtuner/model/modules/dispatch/qwen2.py#L266
xtuner Qwen2的代码里面Qwen2RotaryEmbedding.forward接口接收seq_len参数
![image](https://github.com/user-attachments/assets/7dcae36e-74a2-4c4a-8cca-f037a95e1c6a)
Qwen2RotaryEmbedding是从transformers里面import的，但是transformers从4.45版本以上该接口不接收seq_len参数
![image](https://github.com/user-attachments/assets/68a248a8-4a50-4320-8601-4fab1cd82c8a)
换成了position_ids，因此用xtuner训练模型会报错
![image](https://github.com/user-attachments/assets/16823297-7437-4195-812f-53b311e35b1a)

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

transformers-4.45版本以上Qwen2RotaryEmbedding.forward接口报错

Description

Contributor guide

Assessment

Get new issues in your inbox