apple / apple/coremltools

torch.nn.functional.grid_sample for volumetric input/ Rank 5 input and grid

Open
#2,284 3 comments 0 reactions 0 assignees View on GitHub
Core ML Framework feature request
Dominant language
Python
Stars
5.4k
Forks
850
Avg merge
4d 5h
Merged PRs (30d)
10

Description

## 🌱 Describe your Feature Request
support torch.nn.function.grid_sample for 5d Volumetric input by supporting 5d input for `mb.affine and mb.resample` for rank >=4 :https://pytorch.org/docs/stable/generated/torch.nn.functional.grid_sample.html

## How can this feature be used?
Please provide some examples where this feature can be used.
For video generation models features on device

## Describe alternatives you've considered
currently grid_sampler is supported for rank 4 tensors. it erros out in resample op mb.resample
`input "x" to the "resample" op must be a rank 4 tensor. Got rank 5 tensor of shape (22, 4, c,h,w)`

please support to get mb.affine and mb.resample for rank 5 inputs

Contributor guide

Open the contributing guide

Research direction

Start by tracing the mb.affine and mb.resample implementations and the rank validation that produces the reported error. Compare the supported rank-4 behavior with the PyTorch grid_sample volumetric input and grid requirements. Done means rank-5 inputs work for the requested operations and video-generation conversion examples are covered by validation.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, pytorch
Domain
machine-learning
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.