[Optimization] GridSample is slow and is making InternImage run at less than 1 fps
Nobody has claimed this yet.
- Dominant language
- C++
- Stars
- 333
- Forks
- 150
- Avg merge
- 4d 19h
- Merged PRs (30d)
- 54
Description
DOR (Definition of Ready)
GridSample's ONNX-parser decomposition (src/onnx/parse_gridsample.cpp) is a performance bottleneck: for mode="linear", concat+gathernd index materialization measured at 78 ms. A gridsample operator + GPU JIT kernel was developed in a draft PR and lands this gap for mode="linear" in this draft pr. Precedent and file layout are established; remaining work is extending the same operator to mode="nearest" and mode="cubic", all three padding_mode values, and both align_corners settings. Volumetric (5D) GridSample is explicitly rejected by the parser (gridsample_volumetric_*_test) and is out of scope.
Description
As a MIGraphX user compiling models with instance segmentation heads (e.g. Mask R-CNN mask paste), I want GridSample to run as a single fused GPU kernel for all supported modes (nearest, linear, cubic × zeros/border/reflection × align_corners), not just linear, so that no GridSample usage still falls back to the index-materializing decomposition in parse_gridsample.cpp
DOD (Definition of Done)
- gridsample operator supports mode ∈ {nearest, linear, cubic} (already: linear; add: nearest, cubic).
- GPU JIT kernel handles all three modes; nearest has no interpolation (single tap), cubic computes 16 taps (4×4) inline — no gathernd/concat for any of them.
- Perf comparison (old vs. new) reported for at least one shape per mode.
- Correctness verified against torch.nn.functional.grid_sample for nearest and bicubic, same method as the linear PR.
- CHANGELOG.md entry added.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reviewing the draft PR and src/onnx/parse_gridsample.cpp, then inspect the gridsample_volumetric_*_test cases to confirm the 5D exclusion. Extend the established approach for the remaining modes, padding modes, and align_corners settings, compare performance by mode, verify against torch.nn.functional.grid_sample, and add a CHANGELOG.md entry.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp, pytorch
- Domain
- backend, machine-learning, performance
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Clearly specified
- Newbie friendliness
- 25/100