Help Needed - Bilinear RuntimeError: Expected object of scalar type Half but got scalar type Float for argument #2 'mat2'

I am getting the same issue as #302 . can you help?

i'm using the latest build from apex.
Python 3.7.0
torch 1.2.0
transformers 2.1.1

Here is the sample code:

model, optimizer = amp.initialize(model, optimizer, opt_level=O1, loss_scale=1)
model = torch.nn.DataParallel(model)

class BertApexBug(m.BertPreTrainedModel):
    def __init__(self, config):
        super(BertApexBug, self).__init__(config)
        self.bert = m.BertModel(config)
        self.bi = nn.Bilinear(config.hidden_size, config.hidden_size, 1)
        self.init_weights()

    def forward(self, input_ids, token_type_ids=None, attention_mask=None, labels=None, attention_oracle_mask=None):

        flat_input_ids = input_ids.view(-1, input_ids.size(-1))  
        flat_token_type_ids = token_type_ids.view(-1, token_type_ids.size(-1))
        flat_attention_mask = attention_mask.view(-1, attention_mask.size(-1))

        outputs = self.bert(flat_input_ids, flat_token_type_ids, flat_attention_mask)
        pooled_output = outputs[1]

        bi = self.bi(pooled_output,pooled_output)

        return bi

Original Traceback (most recent call last):
  File "/specific/netapp5_2/gamir/adi/miniconda3/envs/tr_py37/lib/python3.7/site-packages/torch/nn/parallel/parallel_apply.py", line 60, in _worker
    output = module(*input, **kwargs)
  File "/specific/netapp5_2/gamir/adi/miniconda3/envs/tr_py37/lib/python3.7/site-packages/torch/nn/modules/module.py", line 547, in __call__
    result = self.forward(*input, **kwargs)
  File "/specific/netapp5_2/gamir/adi/git/common-sense-distillnet/openbook_bert_models.py", line 170, in forward
    bi = self.bi(pooled_output,pooled_output)
  File "/specific/netapp5_2/gamir/adi/miniconda3/envs/tr_py37/lib/python3.7/site-packages/torch/nn/modules/module.py", line 547, in __call__
    result = self.forward(*input, **kwargs)
  File "/specific/netapp5_2/gamir/adi/miniconda3/envs/tr_py37/lib/python3.7/site-packages/torch/nn/modules/linear.py", line 155, in forward
    return F.bilinear(input1, input2, self.weight, self.bias)
  File "/specific/netapp5_2/gamir/adi/miniconda3/envs/tr_py37/lib/python3.7/site-packages/torch/nn/functional.py", line 1380, in bilinear
    return torch.bilinear(input1, input2, weight, bias)
**RuntimeError: Expected object of scalar type Half but got scalar type Float for argument #2 'mat2'**

Thanks!

Start by comparing this report with issue #302 and reproducing the sample using Python 3.7.0, torch 1.2.0, transformers 2.1.1, and the latest Apex build. Trace the amp.initialize and torch.nn.Bilinear path shown in the traceback; done means establishing a reproducible diagnosis and documenting or testing the supported behavior.