modelscope/ms-swift

View on GitHub

Use PEFT or Full-parameter to CPT/SFT/DPO/GRPO 600+ LLMs (Qwen3.6, DeepSeek-V4, GLM-5.1, InternLM3, Llama4, ...) and 300+ MLLMs (Qwen3-VL, Qwen3-Omni, InternVL3.5, Ovis2.5, GLM4.5v, Gemma4, Llava, Phi4, ...) (AAAI 2025).

Stars: 15.7k
Forks: 1.7k
Open beginner issues: 0
Indexed issues: 473
Avg merge: 1d 16h
Merged PRs (30d): 136

Dominant language: Python
License: Apache-2.0
Last GitHub push: Sep 20, 2026
Latest indexed: Sep 20, 2026
Contributing guide: Contributing guide
Code of conduct: Code of conduct
Beginner labels: good first issue

Beginner friendly Everything

Open All

473 open issues indexed

[Bug] Qwen3.5 MoE multimodal DPO drops output_router_logits and crashes with aux_loss=None Open

Difficulty 2/5 1-3 hours Newbie friendliness 75/100

modelscope/ms-swift#10189 · 1 comment · Opened Sep 17, 2026
[Bug] Negative JSD loss in Megatron GKD at beta=0.5; FP32 gives a positive result on the same inputs Open

bug

Difficulty 4/5 3-5 days Newbie friendliness 70/100

modelscope/ms-swift#10156 · Opened Sep 16, 2026
NPU环境微调qwen3-embedding的正确方式 Open

question

Difficulty 4/5 3-5 days Newbie friendliness 45/100

modelscope/ms-swift#10124 · 1 comment · Opened Sep 14, 2026
[RLHF] DAPO/CISPO/FIPO 的 loss 在梯度累积下被额外除以 gradient_accumulation_steps Open

Difficulty 3/5 1-2 days Newbie friendliness 25/100

modelscope/ms-swift#10117 · 2 comments · Opened Sep 12, 2026
优化 npu_swiglu_forward：拼接投影权重而不是拼接激活以降低内存并提升 NPU 性能 Open

question

modelscope/ms-swift#10108 · 2 comments · 1 assignee · Opened Sep 11, 2026
`fix(megatron): align on-policy resume offset and sampler seed with rollout` Open

bug

Difficulty 4/5 3-5 days Newbie friendliness 65/100

modelscope/ms-swift#10099 · Opened Sep 10, 2026
`fix(gkd): defer full-vocabulary local teacher logits to forward_step` Open

bug

Difficulty 3/5 1-2 days Newbie friendliness 68/100

modelscope/ms-swift#10097 · Opened Sep 10, 2026
`decode_base64_to_image_file` fails on CMYK images — silently drops an entire dataset (e.g. SEEDBench_IMG_KO) Open

bug

Difficulty 2/5 1-3 hours Newbie friendliness 84/100

modelscope/ms-swift#10095 · 1 comment · Opened Sep 10, 2026
Support Top-k MOPD loss Open

enhancement

Difficulty 4/5 3-5 days Newbie friendliness 50/100

modelscope/ms-swift#10079 · Opened Sep 9, 2026
[GKD] `--beta` interpolation is discontinuous at the endpoints: the generalized JSD collapses to 0 as β→0/1, contradicting the documented "degenerates to forward/reverse KL" Open

question

Difficulty 3/5 1-2 days Newbie friendliness 65/100

modelscope/ms-swift#10078 · 1 comment · Opened Sep 9, 2026
[Bug] Teacher template encoding fails on Agent tool-call samples during GKD training Open

bug

Difficulty 4/5 3-5 days Newbie friendliness 25/100

modelscope/ms-swift#10077 · Opened Sep 9, 2026
GRPOTrainer rollout time limited to 10min Open

bug

modelscope/ms-swift#10065 · 1 assignee · Opened Sep 8, 2026
[RFC] Reward model (value-head RM) support in infer/deploy via vLLM Open

Difficulty 5/5 Over a week Newbie friendliness 42/100

modelscope/ms-swift#10058 · 1 reaction · Opened Sep 7, 2026
About qwen3.5 mtp sft, multi-modality data Open

question

Difficulty 4/5 3-5 days Newbie friendliness 38/100

modelscope/ms-swift#10057 · 1 comment · Opened Sep 7, 2026
support for deepseek v4 flash vision exp Open

enhancement

modelscope/ms-swift#10053 · 1 assignee · Opened Sep 7, 2026
Bug: unexpected behavior with special characters in input Open

Difficulty 3/5 1-2 days Newbie friendliness 45/100

modelscope/ms-swift#10024 · 2 comments · Opened Sep 1, 2026
Image augmentation is a bit tricky, tied to models, and hard to do for training data only Open

question

Difficulty 5/5 Over a week Newbie friendliness 35/100

modelscope/ms-swift#10014 · 2 comments · Opened Aug 31, 2026
[NPU][Megatron GRPO] 开启 PP 后 P2P 通信在 HcclGroupEnd 报错并导致进程崩溃 Open

bug

Difficulty 5/5 Over a week Newbie friendliness 35/100

modelscope/ms-swift#10013 · 1 comment · Opened Aug 31, 2026
Data containing multiple images is not supported for the Qwen3.5 series. Open

bug

Difficulty 4/5 3-5 days Newbie friendliness 35/100

modelscope/ms-swift#10006 · 1 comment · Opened Aug 30, 2026
Megatron SFT full finetune breaks tool calling (refuses + hallucinates) while LoRA preserves it — Qwen3.6-27B on Ascend 910B3 Open

Difficulty 4/5 3-5 days Newbie friendliness 35/100

modelscope/ms-swift#10003 · 1 comment · Opened Aug 28, 2026
qwen3.8-27B 几个像素等于一个token，图片大小和token数量如何映射计算的？ Open

question

Difficulty 4/5 3-5 days Newbie friendliness 35/100

modelscope/ms-swift#9970 · 2 comments · Opened Aug 23, 2026
[RFC] Support M2PO for stale-rollout GRPO Open

Difficulty 5/5 Over a week Newbie friendliness 35/100

modelscope/ms-swift#9954 · 4 comments · Opened Aug 20, 2026
[Question] Best Practices for Multi-Turn Long-Context GRPO Training Open

question

Difficulty 5/5 Over a week Newbie friendliness 25/100

modelscope/ms-swift#9942 · 2 comments · Opened Aug 19, 2026
[Bug]: Megatron arg mtp_shared_weights is defined but never consumed (no effect) Open

Difficulty 4/5 3-5 days Newbie friendliness 55/100

modelscope/ms-swift#9936 · 1 comment · Opened Aug 18, 2026
关于多图 grounding 的任务 Open

question

Difficulty 4/5 3-5 days Newbie friendliness 35/100

modelscope/ms-swift#9906 · 3 comments · Opened Aug 13, 2026
Support documented FSDP2 + sequence/context parallel (SP/CP) combo for long-context QLoRA; combining flags on 2 GPUs hits CUDA illegal memory access Open

bug

modelscope/ms-swift#9895 · 1 comment · 1 assignee · Opened Aug 12, 2026
ms-swift支持读写数据湖 Open

enhancement question

Difficulty 5/5 Over a week Newbie friendliness 25/100

modelscope/ms-swift#9890 · 1 comment · Opened Aug 11, 2026
opsd训练 The client socket has timed out after 300000ms while trying to connect to xxxxx:51216 Open

question

Difficulty 4/5 3-5 days Newbie friendliness 28/100

modelscope/ms-swift#9878 · 5 comments · Opened Aug 10, 2026
qwen3.5-4b模型对于padding_free的支持问题 Open

bug

Difficulty 4/5 3-5 days Newbie friendliness 42/100

modelscope/ms-swift#9845 · 3 comments · Opened Aug 4, 2026
DeepSeek-v4-flash-0731 sft Open

question

Difficulty 4/5 3-5 days Newbie friendliness 45/100

modelscope/ms-swift#9836 · 13 comments · Opened Aug 3, 2026
Issues with the MS-Swift Pipeline Layout Feature When Used with MTP Open

bug

Difficulty 4/5 3-5 days Newbie friendliness 48/100

modelscope/ms-swift#9823 · 1 comment · Opened Jul 30, 2026
支持任意 ViT Encoder 与任意 LLM 自由组合构建多模态模型 Open

enhancement

Difficulty 5/5 Over a week Newbie friendliness 30/100

modelscope/ms-swift#9813 · 2 comments · 1 reaction · Opened Jul 29, 2026
训练多模态大模型的时候如果通过传入bbox或者mask在attention计算的时候，让模型更加关注页面的某一区域细节信息 Open

bug

Difficulty 5/5 Over a week Newbie friendliness 25/100

modelscope/ms-swift#9807 · Opened Jul 28, 2026
sft 垂域文生文训练中断 Open

bug

Difficulty 4/5 3-5 days Newbie friendliness 35/100

modelscope/ms-swift#9788 · 2 comments · Opened Jul 22, 2026
A800 训 DSv4-Flash OOM：CSA 无 SM80 kernel，求绕过方案 Open

question

Difficulty 5/5 Over a week Newbie friendliness 35/100

modelscope/ms-swift#9769 · 1 comment · Opened Jul 20, 2026
Support use_logits_to_keep with sequence parallelism for long-context SFT Open

Difficulty 5/5 Over a week Newbie friendliness 45/100

modelscope/ms-swift#9765 · Opened Jul 19, 2026
qwen3.5和qwen3.6 grpo训练 wandb 奖励函数问题 Open

bug

Difficulty 4/5 3-5 days Newbie friendliness 45/100

modelscope/ms-swift#9764 · 2 comments · Opened Jul 18, 2026
Unauthenticated SSRF via multimodal image_url/audio/video in swift deploy (no URL filtering) Open

bug

Difficulty 4/5 3-5 days Newbie friendliness 48/100

modelscope/ms-swift#9740 · Opened Jul 13, 2026
diffusion gemma 微调时数据输出tokens的数量可否支持 > config 中的 256 tokens？ Open

enhancement

Difficulty 4/5 3-5 days Newbie friendliness 45/100

modelscope/ms-swift#9739 · 2 comments · Opened Jul 13, 2026
Support SAO (Single-Rollout Asynchronous Optimization) Open

enhancement

modelscope/ms-swift#9717 · 1 assignee · Opened Jul 10, 2026
GLM-5.2微调 Open

question

Difficulty 4/5 3-5 days Newbie friendliness 25/100

modelscope/ms-swift#9707 · 2 comments · Opened Jul 8, 2026
Qwen3-VL 多标签序列分类（seq_cls, num_labels>2, multi_label_classification）训练后无法用 vLLM 部署 Open

bug

Difficulty 4/5 3-5 days Newbie friendliness 48/100

modelscope/ms-swift#9704 · 1 comment · Opened Jul 8, 2026
Qwen3-Reranker 训练数据集格式到底是啥 Open

bug

Difficulty 4/5 3-5 days Newbie friendliness 32/100

modelscope/ms-swift#9702 · 1 comment · Opened Jul 8, 2026
Packing+多模态数据使用deepspeed微调Qwen3.6-35B-A3B发生数据预处理死锁 Open

bug

Difficulty 4/5 3-5 days Newbie friendliness 48/100

modelscope/ms-swift#9701 · 2 comments · Opened Jul 7, 2026
qwen3.6-35b DPO开启序列并行（--sequence_parallel_size 4）报错： torch.AcceleratorError: CUDA error: an illegal memory access was encountered Open

bug

Difficulty 4/5 3-5 days Newbie friendliness 35/100

modelscope/ms-swift#9692 · 1 comment · Opened Jul 7, 2026
zero3_offload模式下，DataLoader进程由于Fork + COW 机制导致大量内存消耗并OOM Open

bug

Difficulty 4/5 3-5 days Newbie friendliness 42/100

modelscope/ms-swift#9687 · Opened Jul 6, 2026
关于训练速度越来越快的疑问以及为何在接近结束时会疑似因为显存不够报错 Open

question

Difficulty 4/5 3-5 days Newbie friendliness 35/100

modelscope/ms-swift#9681 · 2 comments · 1 reaction · Opened Jul 4, 2026
请问目前ms-swift支持qwen3.5的持续预训练吗？ Open

question

Difficulty 5/5 Over a week Newbie friendliness 25/100

modelscope/ms-swift#9676 · 3 comments · Opened Jul 3, 2026
gemma-4-26b-a4b-it使用megatrons训练有显存OOM问题 Open

bug

Difficulty 4/5 3-5 days Newbie friendliness 38/100

modelscope/ms-swift#9673 · Opened Jul 2, 2026
swift train ↔ vLLM rollout multimodal alignment Open

Difficulty 4/5 3-5 days Newbie friendliness 35/100

modelscope/ms-swift#9668 · Opened Jun 30, 2026

modelscope/ms-swift

Get new issues in your inbox