Convert PyTorch FasterRCNN to CoreML fails

## 🐞Describe the bug
Trying to convert PyTorch FasterRCNN model to CoreML, but getting runtime error `temporary: the only valid use of a module is looking up an attribute but found = prim::SetAttr[name="_has_warned"](%self, %6)`

## Trace
Traceback (most recent call last):
File "/home/harut/Projects/Python/venvs/main.py", line 20, in
main()
File "/home/harut/Projects/Python/venvs/main.py", line 13, in main
mlmodel = coremltools.converters.convert(
File "/home/harut/Projects/Python/venvs/venv/lib/python3.8/site-packages/coremltools/converters/_converters_entry.py", line 292, in convert
proto_spec = _convert(
File "/home/harut/Projects/Python/venvs/venv/lib/python3.8/site-packages/coremltools/converters/mil/converter.py", line 120, in _convert
prog = frontend_converter(model, **kwargs)
File "/home/harut/Projects/Python/venvs/venv/lib/python3.8/site-packages/coremltools/converters/mil/converter.py", line 62, in __call__
return load(*args, **kwargs)
File "/home/harut/Projects/Python/venvs/venv/lib/python3.8/site-packages/coremltools/converters/mil/frontend/torch/load.py", line 73, in load
converter = TorchConverter(torchscript, inputs, outputs, cut_at_symbols)
File "/home/harut/Projects/Python/venvs/venv/lib/python3.8/site-packages/coremltools/converters/mil/frontend/torch/converter.py", line 140, in __init__
raw_graph, params_dict = self._expand_and_optimize_ir(self.torchscript)
File "/home/harut/Projects/Python/venvs/venv/lib/python3.8/site-packages/coremltools/converters/mil/frontend/torch/converter.py", line 328, in _expand_and_optimize_ir
graph, params = _torch._C._jit_pass_lower_graph(
RuntimeError:
temporary: the only valid use of a module is looking up an attribute but found = prim::SetAttr[name="_has_warned"](%self, %6)
:

## To Reproduce
```
import torch
import torchvision
import coremltools
model = torchvision.models.detection.fasterrcnn_resnet50_fpn(pretrained=True)
model.eval()
scripted_model = torch.jit.script(model)

mlmodel = coremltools.converters.convert(
scripted_model,
inputs=[coremltools.TensorType(shape=(1, 3, 512, 512))],
)
```

## System environment (please complete the following information):
- coremltools version: 4.0b1
- OS: Ubuntu 20.04 LTS
- macOS version (if applicable):
- XCode version (if applicable):
- How you install python (anaconda, virtualenv, system): Virtualenv
- python version: 3.8
- any other relevant information:
- [pip3] numpy==1.19.0
- [pip3] torch==1.5.1
- [pip3] torchvision==0.6.1