OptimalScale / OptimalScale/LMFlow

Use `dtype` instead of deprecated `torch_dtype` for transformers >= 4.56

Open Beginner friendly
#977 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
8.5k
Forks
822
PR merge metrics
No merged PRs in 30d

Description

In contrib/rlhflow/reward_modeling.py line 44, AutoModelForSequenceClassification.from_pretrained is called with torch_dtype=torch.bfloat16:

model = AutoModelForSequenceClassification.from_pretrained(
    model_args.model_name_or_path, num_labels=1, torch_dtype=torch.bfloat16
)

The torch_dtype keyword argument was deprecated in transformers 4.56 (PR #39782) and replaced by dtype. On transformers 4.56+ this call emits a DeprecationWarning, and the argument will be removed in a future release, breaking the script.

Suggested fix: choose the keyword based on the installed transformers version with packaging.version:

import transformers
from packaging.version import Version

def _dtype_kwargs(dtype):
    """`dtype` keyword of `from_pretrained` exists since transformers 4.56 (PR #39782);
    older versions use `torch_dtype`."""
    if Version(transformers.__version__) >= Version("4.56"):
        return {"dtype": dtype}
    return {"torch_dtype": dtype}

model = AutoModelForSequenceClassification.from_pretrained(
    model_args.model_name_or_path, num_labels=1, **_dtype_kwargs(torch.bfloat16)
)

This keeps compatibility with transformers < 4.56 and stops the deprecation warning on 4.56+.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in contrib/rlhflow/reward_modeling.py at line 44 and inspect the from_pretrained call and its surrounding imports. Check the installed transformers version handling described in the issue, then update the keyword selection so versions before 4.56 remain compatible and 4.56+ no longer emit the deprecation warning. Verify by running the reward-modeling script with the supported transformers versions.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, pytorch
Domain
machine-learning
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
78/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.