google / google/paxml

How to do gradient accumulation?

Open
#104 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
560
Forks
72
No merged PRs in 30d

Description

I couldnt find much info on how to do gradient accumulation when training with gpus?

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.