apple / apple/coremltools

Support upsample_trilinear3d

Open
#1,782 3 comments 2 reactions 0 assignees View on GitHub
feature request missing layer type PyTorch (traced)
Dominant language
Python
Stars
5.4k
Forks
850
Avg merge
4d 5h
Merged PRs (30d)
10

Description

- Name of layer type: upsample_trilinear3d
- Is this a PyTorch or a TensorFlow layer type: PyTorch
- Your version of coremltools: 6.2
- Your version of PyTorch/TensorFlow: 1.12.1
- Impact of supporting this layer type. Why is adding support for this layer type important? Is it necessary to support a popular model or use case?
Trilinear 3d upsampling is commonly used in 3d networks, volumetric rendering, 3d look-up tables and many other cases. Pytorch supports `trilinear` mode in `torch.nn.functional.interpolate`, `torch.nn.Upsample`.

Example to reproduce:
```
import torch
import coremltools as ct
import torch.nn.functional as F

class Net(torch.nn.Module):

def forward(self, x):
return F.interpolate(x, scale_factor=2.0, mode="trilinear")

class Net2(torch.nn.Module):

def __init__(self) -> None:
super().__init__()
self.upsample3d = torch.nn.Upsample(scale_factor=2.0, mode="trilinear")

def forward(self, x):
return self.upsample3d(x)

input_tensor = torch.zeros([1, 8, 16, 16, 16], dtype=torch.float32)

# Check torch.nn.functional.interpolate
torch_model = Net()
traced_model = torch.jit.trace(torch_model, input_tensor)

model_ct = ct.convert(traced_model,
inputs=[ct.TensorType(shape=input_tensor.shape)])

# Check torch.nn.Upsample
torch_model = Net2()
traced_model = torch.jit.trace(torch_model, input_tensor)

model_ct = ct.convert(traced_model,
inputs=[ct.TensorType(shape=input_tensor.shape)])

```

Error message:
```
RuntimeError: PyTorch convert function for op 'upsample_trilinear3d' not implemented.
```

Contributor guide

Open the contributing guide

Research direction

Start by running the supplied conversion examples for torch.nn.functional.interpolate and torch.nn.Upsample with mode="trilinear" to reproduce the unsupported upsample_trilinear3d error. Trace the PyTorch conversion path for that operation. Done means both examples convert successfully without the reported error.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, pytorch
Domain
machine-learning, tooling
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.