modelscope / modelscope/ms-swift

ms-swift 4.3.0 中如何注册自定义Loss呢

Open
#9,437 3 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

question
Dominant language
Python
Stars
15.7k
Forks
1.7k
Avg merge
1d 16h
Merged PRs (30d)
136

Description

Checklist / 检查清单
  • I have searched existing issues, and this is a new question or discussion topic. / 我已经搜索过现有的 issues,确认这是一个新的问题与讨论。
Question Description / 问题描述

文档中写道:

Loss的mapping文件可以参考[这里](https://github.com/modelscope/ms-swift/blob/main/swift/loss/mapping.py)。 swift支持自定义loss(当前只支持sft/pretrain/reranker/embedding任务),注册后在训练时设置--loss_type <loss-name>使用你定制的loss方法。

自定义Loss需继承自BaseLoss,并实现__call__方法,返回标量Tensor。你可以参考[CustomCrossEntropyLoss](https://github.com/modelscope/ms-swift/blob/0d7c9f5bc0e7e7d67d914ce6edeb9ce24f60746f/swift/loss/causal_lm.py#L5)进行定制。例如:

class CustomLoss(BaseLoss):

    def __call__(self, outputs, labels, **kwargs) -> torch.Tensor:
        pass

但是未提及如何在外部注册(可能是我没找到?只有一些旧版本文档中有)这个loss函数到swift中来支持命令行调用,目前是直接在swift源码loss中实现的损失函数再在mapping中指定,也可以做到

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with swift/loss/mapping.py and the linked CustomCrossEntropyLoss in swift/loss/causal_lm.py, then inspect the existing documentation for custom Loss registration. Verify how loss_type is resolved for CLI training and document the supported external-registration workflow with a minimal example; the documentation should make the command-line usage clear.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, pytorch
Domain
documentation
Issue type
Documentation
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
58/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.