[eager_fail_to_run ] mobilenet_v2_quantized_qat, eager run failed on dtype mismatch

Running from pytorch:

python benchmarks/dynamo/torchbench.py  --only mobilenet_v2_quantized_qat --accuracy --no-translation-validation --training --amp --backend inductor --device cuda --output out.csv

Error:

cuda train mobilenet_v2_quantized_qat
Traceback (most recent call last):
  File "/data/users/ivankobzarev/a/pytorch/benchmarks/dynamo/common.py", line 2744, in validate_model
    self.model_iter_fn(model, example_inputs)
  File "/data/users/ivankobzarev/a/pytorch/benchmarks/dynamo/torchbench.py", line 453, in forward_and_backward_pass
    pred = mod(*cloned_inputs)
  File "/data/users/ivankobzarev/a/pytorch/torch/fx/graph_module.py", line 824, in call_wrapped
    return self._wrapped_call(self, *args, **kwargs)
  File "/data/users/ivankobzarev/a/pytorch/torch/fx/graph_module.py", line 400, in __call__
    raise e
  File "/data/users/ivankobzarev/a/pytorch/torch/fx/graph_module.py", line 387, in __call__
    return super(self.cls, obj).__call__(*args, **kwargs)  # type: ignore[misc]
  File "/data/users/ivankobzarev/a/pytorch/torch/nn/modules/module.py", line 1749, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/data/users/ivankobzarev/a/pytorch/torch/nn/modules/module.py", line 1760, in _call_impl
    return forward_call(*args, **kwargs)
  File "<eval_with_key>.3", line 207, in forward
    activation_post_process_101 = self.activation_post_process_101(classifier_1);  classifier_1 = None
  File "/data/users/ivankobzarev/a/pytorch/torch/nn/modules/module.py", line 1749, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/data/users/ivankobzarev/a/pytorch/torch/nn/modules/module.py", line 1760, in _call_impl
    return forward_call(*args, **kwargs)
  File "/data/users/ivankobzarev/a/pytorch/torch/ao/quantization/fake_quantize.py", line 408, in forward
    return torch.fused_moving_avg_obs_fake_quant(
RuntimeError: expected scalar type Float but found Half

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/data/users/ivankobzarev/a/pytorch/benchmarks/dynamo/common.py", line 4857, in run
    ) = runner.load_model(
  File "/data/users/ivankobzarev/a/pytorch/benchmarks/dynamo/torchbench.py", line 372, in load_model
    self.validate_model(model, example_inputs)
  File "/data/users/ivankobzarev/a/pytorch/benchmarks/dynamo/common.py", line 2746, in validate_model
    raise RuntimeError("Eager run failed") from e
RuntimeError: Eager run failed

Start by reproducing the command in benchmarks/dynamo/torchbench.py and follow validation through benchmarks/dynamo/common.py. Inspect the failing path in torch/ao/quantization/fake_quantize.py; done means mobilenet_v2_quantized_qat completes the eager training run without the Float-versus-Half dtype error.