InternLM / InternLM/xtuner

chatglm3-6B 微调报错

Open
#796 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
5.2k
Forks
448
Avg merge
3d 15h
Merged PRs (30d)
26

Description

现在新版本xtuner增加了dispatch后,不支持chatglm3-6b的微调了吗
File "/mnt/afs/xtuner/xtuner/model/sft.py", line 93, in __init__
dispatch_modules(self.llm, use_varlen_attn=use_varlen_attn)
File "/mnt/afs/xtuner/xtuner/model/modules/dispatch/__init__.py", line 266, in dispatch_modules
check(type(model).__name__)
File "/mnt/afs/xtuner/xtuner/model/modules/dispatch/__init__.py", line 261, in check
assert TRANSFORMERS_VERSION >= LOWEST_TRANSFORMERS_VERSION[
KeyError: 'ChatGLMForConditionalGeneration'

看了一下确实没有chatglm
LOWEST_TRANSFORMERS_VERSION = dict(
InternLM2ForCausalLM=digit_version('4.36'),
InternLMForCausalLM=digit_version('4.36'),
LlamaForCausalLM=digit_version('4.36'),
Phi3ForCausalLM=digit_version('4.39'),
MistralForCausalLM=digit_version('4.36'),
# Training mixtral with lower version may lead to nccl timeout
# Refer to https://github.com/microsoft/DeepSpeed/issues/5066
MixtralForCausalLM=digit_version('4.40'),
CohereForCausalLM=digit_version('4.40'),
Qwen2ForCausalLM=digit_version('4.39'),
Qwen2MoeForCausalLM=digit_version('4.40'),
DeepseekV2ForCausalLM=digit_version('4.40'),
)

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.