modelscope / modelscope/ms-swift

训练LongCat-Flash-Omni sft报错

Open
#7,259 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

more models stale
Dominant language
Python
Stars
15.7k
Forks
1.7k
Avg merge
1d 16h
Merged PRs (30d)
136

Description

如题,训练LongCat-Flash-Omni sft报错。

model_path=model/LongCat-Flash-Omni

megatron sft \ --model ${model_path} \ --load_safetensors true \ --save_safetensors true \ --dataset 'data/base_user_profile.json' \ 'data/online.json' \ --val_dataset 'paras2s_1217.json' \ --tensor_model_parallel_size 4 \ --pipeline_model_parallel_size 2 \ --expert_model_parallel_size 4 \ --context_parallel_size 1 \ --moe_grouped_gemm true \ --moe_shared_expert_overlap true \ --moe_aux_loss_coeff 0 \ --micro_batch_size 1 \ --global_batch_size 16 \ --freeze_llm false \ --freeze_vit true \ --freeze_aligner true \ --recompute_granularity full \ --recompute_method uniform \ --recompute_num_layers 1 \ --max_epochs 2 \ --finetune true \ --rpo_alpha 0.1 \ --cross_entropy_loss_fusion true \ --lr 1e-6 \ --lr_warmup_fraction 0.00 \ --min_lr 1e-7 \ --save ${output_path} \ --eval_interval 100 \ --save_interval 100 \ --max_length 8192 \ --packing true \

报错:

^^^^^^^^^^^^^^^ File "<string>", line 313, in __init__ File "ms_swift_1109/swift/megatron/argument/train_args.py", line 42, in __post_ init__ assert self.megatron_model_meta is not None, f'Model: {self.model} is not supported.' ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ AssertionError: Model: model/LongCat-Flash-Omni is not supported. super().__post_init__() File "ms_swift_1109/swift/megatron/argument/megatron_base_args.py", line 22, in __post_init__ assert self.megatron_model_meta is not None, f'Model: {self.model} is not supported.' ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ AssertionError: Model: model/LongCat-Flash-Omni is not supported.

LongCat-Flash-Omni 模型:https://github.com/meituan-longcat/LongCat-Flash-Omni

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with swift/megatron/argument/train_args.py, where the reported assertion rejects model/LongCat-Flash-Omni. Compare the supported-model handling with the LongCat-Flash-Omni repository linked in the issue, then reproduce the provided SFT command. Done means the model is recognized and this training command proceeds past argument validation.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.