A model that concats two inputs is not converted correctly.
- Dominant language
- Python
- Stars
- 5.4k
- Forks
- 850
- Avg merge
- 4d 5h
- Merged PRs (30d)
- 10
Description
## ❓Question
I want to convert an inpainting model with two inputs.
https://github.com/researchmm/AOT-GAN-for-Inpainting
One input is the masked original image (x) and the other input is the mask image (mask).
When converted to torchscript, the model concats the two inputs as intended, as shown in the attached image.
```
x = torch.rand(1, 3, 512, 512)
mask = torch.rand(1, 1, 512, 512)
ts = torch.jit.trace(model, (x,mask))
```

However, when I convert to coreml model with the code below, the conversion itself completes correctly, but the converted mlmodel double joins one input (mask) and ignores the other input (x).
```
import coremltools as ct
mlmodel = ct.convert(ts, inputs=[ct.ImageType(shape=x.shape),ct.ImageType(shape=mask.shape)])
mlmodel.save("inpainting.mlmodel")
```

concat receives x:1 and mask.
The output of the ignored input (x) is x:0, and the mask input (mask) outputs x:1 and mask and is concat.
Is there a way to concat the two inputs as intended by the original model?
Thank you for your work. Best,
Contributor guide
Research direction
Start with the Python conversion call using the traced TorchScript model and the two ImageType inputs, then inspect how the converted model represents the concat inputs. Reproduce the AOT-GAN inpainting case with x and mask; done means both inputs feed the concat instead of x being ignored or mask being duplicated.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- machine-learning, tooling
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100