deepspeedai / deepspeedai/DeepSpeed

Feature request: Ability to disable autocast locally

Open
#2,400 2 comments 6 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement training
Dominant language
Python
Stars
43.1k
Forks
5k
Avg merge
4d 15h
Merged PRs (30d)
112

Description

While training in half precision, it is often desirable to temporarily disable autocast and run code in full precision. Using native torch AMP, this is as simple as:

with torch.cuda.amp.autocast(enabled=False):
    <do stuff without autocasting>

Is it currently possible to accomplish the same thing while using DeepSpeed's fp16 or bfloat16 modes? If not, this would be great to have.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by examining DeepSpeed's fp16 and bfloat16 modes and how they handle autocasting. The issue names no files or tests; done would be a documented or implemented way to disable autocast locally while retaining the requested full-precision behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, pytorch
Domain
machine-learning
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.