Mlpackage of StyleGAN2 gets wrong results. The output of ToRGB module has large difference between mlmodel and mlpackage.

## 🐞Describe the bug

It gets wrong result when the output adds a skip connection value. I tested the difference by absolute mean error. The error is zero when I only test a single ToRGB module. But when I test on the StyleGAN2 model and comment some lines, the error of ToRGB module is around 4e-2, and the error is below 1e-4 before adding the skip connection.

The results generated by mlmodel and mlpackage.

## To Reproduce

ToRGB module (error is around 4e-2):
```python
class ToRGB(nn.Module):
def __init__(self, in_channel, style_dim, upsample=True, blur_kernel=[1, 3, 3, 1], version='v1', fuse=False):
super().__init__()
if upsample:
self.upsample = Upsample(blur_kernel)

self.conv = ModulatedConv2d(in_channel, 3, 1, style_dim, demodulate=False)
self.bias = nn.Parameter(torch.zeros(1, 3, 1, 1))

def forward(self, input, style, skip=None, upsample=True):
out = self.conv(input, style)
out = out + self.bias
if skip is not None:
if upsample:
skip = self.upsample(skip)
out = out + skip
return out
```

Return the skip connection and error is below 1e-4:
```python
class ToRGB(nn.Module):
def __init__(self, in_channel, style_dim, upsample=True, blur_kernel=[1, 3, 3, 1], version='v1', fuse=False):
super().__init__()
if upsample:
self.upsample = Upsample(blur_kernel)

self.conv = ModulatedConv2d(in_channel, 3, 1, style_dim, demodulate=False)
self.bias = nn.Parameter(torch.zeros(1, 3, 1, 1))

def forward(self, input, style, skip=None, upsample=True):
out = self.conv(input, style)
out = out + self.bias
if skip is not None:
if upsample:
skip = self.upsample(skip)
return skip
# out = out + skip
# return out
```

Return the output berfore adding skip connection and error is below 1e-8:
```python
class ToRGB(nn.Module):
def __init__(self, in_channel, style_dim, upsample=True, blur_kernel=[1, 3, 3, 1], version='v1', fuse=False):
super().__init__()
if upsample:
self.upsample = Upsample(blur_kernel)

self.conv = ModulatedConv2d(in_channel, 3, 1, style_dim, demodulate=False)
self.bias = nn.Parameter(torch.zeros(1, 3, 1, 1))

def forward(self, input, style, skip=None, upsample=True):
out = self.conv(input, style)
out = out + self.bias
return out
# if skip is not None:
# if upsample:
# skip = self.upsample(skip)
# return skip
# out = out + skip
# return out
```

## System environment (please complete the following information):
- MacOS
- coremltools == 5.2
- Pytorch==1.9

Start by running the reported StyleGAN2 reproduction with Python, PyTorch 1.9, coremltools 5.2, and macOS, then compare the mlmodel and mlpackage outputs around the ToRGB module. Check the skip connection and upsampling path against the reported absolute mean errors; done means explaining and correcting the discrepancy so the converted output matches the reference.

Written by the indexing model from the issue text.