THUDM / THUDM/slime

SFT for multi-turn tool call data is not supported properly

Open
#1,008 2 comments 2 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
8.5k
Forks
1.3k
Avg merge
5h 36m
Merged PRs (30d)
22

Description

SFT Tool Support Proposal

Problem

  1. When --apply-chat-template is used, the message history (list[dict]) gets converted to a string in data.py. This breaks gen_multi_turn_loss_mask_qwen and other functions in mask_utils.py, which expect a list[dict].

  2. --tool-key requires --apply-chat-template, which triggers the same string conversion issue.

  3. In mask_utils.py, apply_chat_template is called on each message individually without the tools parameter, so tool-related messages (assistant with tool_calls, tool role responses) are not formatted correctly.

Downstream Effects

  • Without these flags: Tool definitions are never added to the conversation, so the model never sees them during training.
  • With these flags: A fatal error is thrown because mask_utils receives a string instead of list[dict].
  • Even if it didn't crash: apply_chat_template without tools= produces incorrect tokenization for tool-calling messages.

Proposal

A non-invasive fix with two changes:

1. data.py - Store tools in metadata

When --tool-key is provided:

  • Extract tools from the data
  • Store them in sample.metadata["tools"]
  • Keep messages as list[dict] (do NOT apply chat template)
2. mask_utils.py - Pass tools to apply_chat_template

Update the mask generation functions to:

  • Accept an optional tools parameter
  • Pass tools to apply_chat_template so tool-calling messages are formatted correctly

Also this fixes Issue #289

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Read data.py and mask_utils.py, starting with the --apply-chat-template and --tool-key handling and the gen_multi_turn_loss_mask_qwen path. Trace how message history and metadata reach mask generation. Done means tool definitions remain available through metadata, messages stay as list[dict], and apply_chat_template receives tools for tool-calling messages without the reported failure.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.