Is there a strategy to solve overflow in forward process
Open
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 9k
- Forks
- 1.5k
- Avg merge
- 2d 4h
- Merged PRs (30d)
- 3
Description
When I use automatic mixed precision for training, I encounter the overflow problem in forward process（not in backward）. Is there a strategy in the amp to solve this problem?
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
The issue reports overflow during the forward process when automatic mixed precision is used for training, but names no files, tests, or entry points. Start by reproducing the forward-pass overflow and reviewing the AMP behavior; done would require identifying and validating a strategy that prevents or handles this overflow.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100