modelscope / modelscope/ms-swift
训练LongCat-Flash-Omni sft报错
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 15.7k
- Forks
- 1.7k
- Avg merge
- 1d 16h
- Merged PRs (30d)
- 136
Description
如题,训练LongCat-Flash-Omni sft报错。
model_path=model/LongCat-Flash-Omni
megatron sft \ --model ${model_path} \ --load_safetensors true \ --save_safetensors true \ --dataset 'data/base_user_profile.json' \ 'data/online.json' \ --val_dataset 'paras2s_1217.json' \ --tensor_model_parallel_size 4 \ --pipeline_model_parallel_size 2 \ --expert_model_parallel_size 4 \ --context_parallel_size 1 \ --moe_grouped_gemm true \ --moe_shared_expert_overlap true \ --moe_aux_loss_coeff 0 \ --micro_batch_size 1 \ --global_batch_size 16 \ --freeze_llm false \ --freeze_vit true \ --freeze_aligner true \ --recompute_granularity full \ --recompute_method uniform \ --recompute_num_layers 1 \ --max_epochs 2 \ --finetune true \ --rpo_alpha 0.1 \ --cross_entropy_loss_fusion true \ --lr 1e-6 \ --lr_warmup_fraction 0.00 \ --min_lr 1e-7 \ --save ${output_path} \ --eval_interval 100 \ --save_interval 100 \ --max_length 8192 \ --packing true \
报错:
^^^^^^^^^^^^^^^ File "<string>", line 313, in __init__ File "ms_swift_1109/swift/megatron/argument/train_args.py", line 42, in __post_ init__ assert self.megatron_model_meta is not None, f'Model: {self.model} is not supported.' ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ AssertionError: Model: model/LongCat-Flash-Omni is not supported. super().__post_init__() File "ms_swift_1109/swift/megatron/argument/megatron_base_args.py", line 22, in __post_init__ assert self.megatron_model_meta is not None, f'Model: {self.model} is not supported.' ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ AssertionError: Model: model/LongCat-Flash-Omni is not supported.
LongCat-Flash-Omni 模型:https://github.com/meituan-longcat/LongCat-Flash-Omni
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with swift/megatron/argument/train_args.py, where the reported assertion rejects model/LongCat-Flash-Omni. Compare the supported-model handling with the LongCat-Flash-Omni repository linked in the issue, then reproduce the provided SFT command. Done means the model is recognized and this training command proceeds past argument validation.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100