BUG: Opt-level 01 not working
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 9k
- Forks
- 1.5k
- Avg merge
- 2d 4h
- Merged PRs (30d)
- 3
Description
I have PyTorch v1.2 and Apex installed. I was able to get opt_level 00 and 03 working. But, opt_level 01 option isn't working. From the debugging I've done, looks like it's crashing at prefetcher = data_prefetcher(train_loader). However, there aren't any error messages to help with debugging.
OUTPUT:
$ python main_amp.py -a resnet50 --b 224 --workers 4 --opt-level O1 data
opt_level = O1
keep_batchnorm_fp32 = None <class 'NoneType'>
loss_scale = None <class 'NoneType'>
CUDNN VERSION: 7602
=> creating model 'resnet50'
Selected optimization level O1: Insert automatic casts around Pytorch functions and Tensor methods.
Defaults for this optimization level are:
enabled : True
opt_level : O1
cast_model_type : None
patch_torch_functions : True
keep_batchnorm_fp32 : None
master_weights : None
loss_scale : dynamic
Processing user overrides (additional kwargs that are not None)...
After processing overrides, optimization options are:
enabled : True
opt_level : O1
cast_model_type : None
patch_torch_functions : True
keep_batchnorm_fp32 : None
master_weights : None
loss_scale : dynamic
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with main_amp.py and the data_prefetcher(train_loader) call, then reproduce the reported command with opt-level O1 and compare it with O0 and O3. Investigate why O1 stops without an error message; done means identifying the failure and adding a verified fix or a diagnostic that exposes it.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100