Fine-tuning bert-base model using mixed precision does not reduce model size

Open
#891 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
25/100
Issue type
Bug
Clarity
Needs clarification
Activity status
Stale
Tech stack
python
Domain
machine-learning

Research direction

No repository file, test, or entry point is named in the report. Start by reproducing fine-tuning of bert-base with Apex O1, O2, and without Apex using the stated PyTorch 1.4 and CUDA 10.2 versions. Done means explaining whether identical saved model sizes are expected and identifying any confirmed Apex behavior or missing warning.

Written by the indexing model from the issue text.

Description

Hello all,

I have fine-tuned bert model with different apex levels i.e. O1, O2, and without apex. At the end, I am having same model size for all the fine-tuned models. Could anyone explain why the model sizes are same for all the apex levels and without apex too?

PS: I do not get any warning related to apex while fine-tuning

Version:

pytorch : 1.4
torch.version.cuda : 10.2
Nvidia cuda version: 10.2
Dominant language
Python
Stars
9k
Forks
1.5k
Avg merge
2d 4h
Merged PRs (30d)
3

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from NVIDIA/apex

All issues in NVIDIA/apex

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.