google-research / google-research/big_vision
bfloat16 Training
Open
more info needed
- Dominant language
- Jupyter Notebook
- Stars
- 3.5k
- Forks
- 227
- PR merge metrics
- No merged PRs in 30d
Description
Thank you for releasing code for these inspiring works!
I tried to use bfloat16 for model parameters, and manually converted images and labels from float32 to bfloat16 before feeding them for training, but noticed that training slowed down by about 3 times. Also, the performance becomes noticeably worse. I'm wondering if it is wrong to use bfloat16 in this way?
Thank you very much for your help.
Contributor guide
Research direction
The issue names no files, tests, or entry points. Start by reviewing the bfloat16 parameter and image/label conversion setup described in the report; the issue does not define a code change or a clear completion criterion.
Written by the indexing model from the issue text.
Assessment
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 20/100