modelscope / modelscope/ms-swift
Question about New model and algorithms support
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 15.7k
- Forks
- 1.7k
- Avg merge
- 1d 16h
- Merged PRs (30d)
- 136
Description
I want to adapt my RL algorithm, specifically designed for DLLM (Diffusion Language Model), using the SDAR-8B-Chat model and its SFT and RL-trained versions. Currently, this model is compatible with the Lmdeploy inference engine, which can be used as a rollout tool. However, I don't know how to adapt it to ms-swift, because for RL algorithms like GRPO, it requires redesigning the attention mechanism and forward process, meaning I need to manually modify the intermediate training details. I want to adapt it to ms-swift because your framework supports lmdeploy rollout. How can I customize this RL training process?
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reviewing ms-swift's GRPO training path and its lmdeploy rollout integration, then identify how the SDAR-8B-Chat model's custom attention and forward process could fit. Done would be a documented customization path or a clear determination that this model requires new framework support.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 20/100