modelscope / modelscope/ms-swift
Qwen3-VL-30B-A3B-Instruct much worse performance after fine-tune compared to 32B and 8B dense
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 15.7k
- Forks
- 1.7k
- Avg merge
- 1d 16h
- Merged PRs (30d)
- 136
Description
@Jintao-Huang We found that the 30B-A3B model performs worse after finetuning compared to 32B and 8B dense with almost exactly the same hyperparameters and default settings recommended. Do we have some tests to verify that there are no bugs.
I am using megatron sft
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the reported megatron sft workflow and compare the Qwen3-VL-30B-A3B fine-tuning configuration with the otherwise similar 32B and 8B dense runs. Reproduce the performance difference and investigate whether a test can distinguish a model-specific regression from expected behavior; done means the cause or a reliable verification test is documented.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100