Training speed does not get faster after using apex
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 9k
- Forks
- 1.5k
- Avg merge
- 2d 4h
- Merged PRs (30d)
- 3
Description
Hi there,
I am on a Ubuntu machine with a V100 using Python3.6.8 + PyTorch 1.4.0 + Torchvision 0.5.0 + CUDA10.1 + CUDNN7.6.5. I installed apex with the command pip install -v --no-cache-dir --global-option="--pyprof" --global-option="--cpp_ext" --global-option="--cuda_ext" ./ .
When I train my model like model, optimizer = amp.initialize(model, optimizer, opt_level='O1') , the training speed has not become faster. What may be the cause?
Any help is appreciated
Thanks
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reproducing the reported training setup with Python 3.6.8, PyTorch 1.4.0, Torchvision 0.5.0, CUDA 10.1, cuDNN 7.6.5, and the shown amp.initialize call at opt_level='O1'. Compare training speed before and after Apex installation and determine whether a reproducible cause or actionable diagnosis can be established.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100