THUDM / THUDM/slime

[Question] I found that ois is numerically equivalent to tis during training

Open
#1,907 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

question
Dominant language
Python
Stars
8.5k
Forks
1.3k
Avg merge
5h 36m
Merged PRs (30d)
22

Description

Your Question

Hi,
I am training Qwen3.5-4B with Slime v0.2.4 with the following config:

  • async
  • update weight step = 2
  • use tis
    When I look at the logged metrics, I found that OIS is numerically equal to TIS during training, which should not happen. Is this a bug or something expected? Thanks in advance for any help.

Here are the figures:

Image Image
What I've Tried

I read through the code and found they should not be equal to each other

Environment (if relevant)
  • slime version: v0.2.4
  • Python version:
  • PyTorch version:
  • CUDA/ROCm version:
  • GPU type and count:
  • OS:
Additional Context

No response

Pre-submission Checklist

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the reported training setup in Slime v0.2.4: async mode, update weight step 2, and TIS enabled. Compare the logged OIS and TIS metrics against the expected distinction; done means determining whether their equality is expected or producing a reproducible bug report with the missing environment details.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.