modelscope / modelscope/ms-swift
是否可以支持KTransformers,优化Moe的微调成本?
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 15.7k
- Forks
- 1.7k
- Avg merge
- 1d 16h
- Merged PRs (30d)
- 136
Description
Describe the feature
看到同类框架 联手 KTransformers 推出降低moe模型训练成本的方案。
原文如下:
实现超大规模模型的 LoRA 微调,彻底解决之前 HuggingFace/Unsloth backend 跑不动、不支持的问题,70G 显存即可LoRA微调 671B 模型!
⭐️核心亮点速览
硬件门槛低:14B 模型微调仅需 6GB GPU 内存,671B 模型总 GPU 内存控制在 70GB 内(可分在多张显卡);
速度效率高:14B 模型微调吞吐量比 HuggingFace 快 75%,671B 模型也能稳定跑 40 token/s;
用着超顺手:沿用 LLaMA-Factory 的原生工作流,不用改一行核心代码,老用户零学习成本;
场景适配强:风格对话、西式翻译、非洲医疗 QA 等场景均验证成功。
🛫后续规划
近期会完善更多模型支持(比如 Qwen3-MoE、Kimi-K2 的微调模板);
优化多 GPU 调度逻辑;进一步降低 CPU 内存占用。
现在大家可以直接在 LLaMA-Factory 里启用 KTransformers backend 体验,相关配置示例和优化规则已同步到双方开源仓库。非常感谢大家的关注和支持,后续会持续迭代,让超大规模模型的微调 / 部署更流畅、更易用!
Paste any useful information
参考文档:2张4090竟能本地微调万亿参数Kimi K2!趋境联合清华北航把算力门槛击穿了
https://mp.weixin.qq.com/s/cNk68HrnBaVDABbsCGl4NQ
Additional context
Add any other context or information here(其他信息可以写在这里)
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
The issue proposes adding a KTransformers backend for MoE fine-tuning, but it names no ms-swift files, tests, or entry points. Start by locating the existing backend and MoE fine-tuning integration points, then compare the referenced KTransformers configuration and optimization rules. Done should include documented support and validation for the proposed workflow.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 20/100