modelscope / modelscope/ms-swift

用swift微调Qwen3模型的function calling功能,如何构建数据集和模型参数以及如何避免过拟合和欠拟合

Open
#9,084 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

question
Dominant language
Python
Stars
15.7k
Forks
1.7k
Avg merge
1d 16h
Merged PRs (30d)
136

Description

Checklist / 检查清单
  • I have searched existing issues, and this is a new question or discussion topic. / 我已经搜索过现有的 issues,确认这是一个新的问题与讨论。
Question Description / 问题描述

这是数据集格式{"tools": [{"name": "query_ledger", "description": "查询电力台账数据。根据站点名称、母线名称、线路名称查询相关的电力设施台账信息。", "parameters": {"type": "object", "properties": {"subName": {"type": "string", "description": "站点名称"}}, "required": ["subName"]}}}], "messages": [{"role": "system", "content": "你是电网助手。先判断是否需要查询台账,再决定是否调用工具,最后给出结论。"}, {"role": "user", "content": "{"alert_content": "2025-08-14 20:19:23 某某站 退出(前置机A报警)", "alert_level": "事故", "signal_id": "", "signal_package": ["2025-08-14 20:19:23 某某站 退出(前置机A报警)"], "station_inside_outer": "1", "throbNum": "0"}"}, {"role": "assistant", "content": "\n某某"沙田站 退出(前置机A报警)"描述关键字"前置机A报警"判断告警类型为四遥中断,四遥中断类型的信号需要调用 query_ledger 查询站点台账进行辅助判断信号的正确性。\n从告警内容中提取站点名称为"某某站",调用 query_ledger 工具查询某某站的台账信息。\n", "tool_calls": [{"name": "query_ledger", "arguments": "{"subName": "沙田站", "busName": "", "lineName": ""}"}]}, {"role": "tool", "content": "[]"}, {"role": "assistant", "content": "经查询某某站不在台账中,说明沙田站不属于调管厂站,不需要关注某某站的任何信号,而沙田站 退出(前置机A报警)告警信号属于某某站不在调管范围,所以这条信号为正常信号。\n\n智能体查询到台账数量为0,说明台账中不存在相关某某站信息,说明某某站不属于调管厂站,不需要关注这个某某站告警信号,结果默认为正常信号。"}]}
这是模型参数/Users/e8900ai/anaconda3/envs/sft/bin/swift sft
--model /Users/e8900ai/LLM_Fine/models/qwen/Qwen3-4B
--tuner_type lora
--dataset /Users/e8900ai/LLM_Fine/models/data/data_update6.jsonl
--dataset /Users/e8900ai/LLM_Fine/models/data/general_agent_dataset.jsonl
--agent_template hermes
--split_dataset_ratio 0.1
--torch_dtype bfloat16
--num_train_epochs 2
--per_device_train_batch_size 4
--per_device_eval_batch_size 6
--learning_rate 1e-5
--gradient_accumulation_steps 4
--eval_steps 50
--save_steps 100
--save_total_limit 3
--logging_steps 10
--max_length 4096
--save_only_model true
--packing false
--attn_impl sdpa
--output_dir /Users/e8900ai/LLM_Fine/models/output4-4B
--lora_rank 8
--lora_alpha 16
--target_modules q_proj, k_proj, v_proj, o_proj
--lora_dropout 0.1
--lr_scheduler_type linear
--do_eval true
--load_best_model_at_end true
--eval_strategy steps
--metric_for_best_model loss
--train_sampling_strategy random
2>&1 | tee /Users/e8900ai/LLM_Fine/models/output4-4B/logs/training_$(date +%Y%m%d_%H%M%S).log

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Review the dataset example and the supplied ms-swift sft command, including the LoRA, sampling, evaluation, and sequence-length settings. No repository file or test is named, so first locate the relevant fine-tuning and tool-calling documentation or entry points. Done means documenting validated dataset and parameter guidance that addresses overfitting and underfitting for this setup.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, swift
Domain
ai, machine-learning
Issue type
Documentation
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.