chatglm3-6B 微调报错
- Dominant language
- Python
- Stars
- 5.2k
- Forks
- 448
- Avg merge
- 3d 15h
- Merged PRs (30d)
- 26
Description
现在新版本xtuner增加了dispatch后,不支持chatglm3-6b的微调了吗
File "/mnt/afs/xtuner/xtuner/model/sft.py", line 93, in __init__
dispatch_modules(self.llm, use_varlen_attn=use_varlen_attn)
File "/mnt/afs/xtuner/xtuner/model/modules/dispatch/__init__.py", line 266, in dispatch_modules
check(type(model).__name__)
File "/mnt/afs/xtuner/xtuner/model/modules/dispatch/__init__.py", line 261, in check
assert TRANSFORMERS_VERSION >= LOWEST_TRANSFORMERS_VERSION[
KeyError: 'ChatGLMForConditionalGeneration'
看了一下确实没有chatglm
LOWEST_TRANSFORMERS_VERSION = dict(
InternLM2ForCausalLM=digit_version('4.36'),
InternLMForCausalLM=digit_version('4.36'),
LlamaForCausalLM=digit_version('4.36'),
Phi3ForCausalLM=digit_version('4.39'),
MistralForCausalLM=digit_version('4.36'),
# Training mixtral with lower version may lead to nccl timeout
# Refer to https://github.com/microsoft/DeepSpeed/issues/5066
MixtralForCausalLM=digit_version('4.40'),
CohereForCausalLM=digit_version('4.40'),
Qwen2ForCausalLM=digit_version('4.39'),
Qwen2MoeForCausalLM=digit_version('4.40'),
DeepseekV2ForCausalLM=digit_version('4.40'),
)
Contributor guide
Assessment
This issue has not been assessed yet.