pytorch / pytorch/executorch

Convnext model fails at runtime on Vulkan

Open
#14,053 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

backend tester module: vulkan
Dominant language
Python
Stars
5k
Forks
1.2k
Avg merge
2d 10h
Merged PRs (30d)
581

Description

🐛 Describe the bug

The convnext_small model from torchvision fails at runtime on the Vulkan backend with "RuntimeError: Exception raised from add_reduce2d_node at /Users/gjcomer/src/executorch/backends/vulkan/runtime/graph/ops/impl/Reduce.cpp:186: (graph.packed_dim_of(in) != reduce_dim1) is false!".

This can be reproduced with the following test case command or standalone script.

python -m executorch.backends.test.suite.runner models --flow vulkan --filter "test_convnext_small_vulkan_float32$"

Standalone repro:

import torch
import torchvision
from executorch.exir import to_edge_transform_and_lower

from executorch.backends.vulkan.partitioner.vulkan_partitioner import VulkanPartitioner

inputs = (torch.randn(1, 3, 144, 144),)
model = torchvision.models.convnext_small().eval()

ep = torch.export.export(model, inputs)

model = to_edge_transform_and_lower(
    torch.export.export(model, inputs),
    partitioner=[VulkanPartitioner()],
).to_executorch()

print("Running model...")
from executorch.extension.pybindings.portable_lib import _load_for_executorch_from_buffer
loaded_model = _load_for_executorch_from_buffer(model.buffer)
loaded_model([*inputs])

Note that running the backend test case requires executorch's python bindings to be built with the Vulkan backend. An example build command is below.

CMAKE_ARGS="-DEXECUTORCH_BUILD_VULKAN=ON" ./install_executorch.sh --editable
Versions

Commit fbda3a9545de747329577bd910086072ec5c7ad1, M1 Mac, using MoltenVK

cc @SS-JIA @manuelcandales @cbilgin

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the Vulkan runtime failure at backends/vulkan/runtime/graph/ops/impl/Reduce.cpp:186, then run the test suite command for test_convnext_small_vulkan_float32 after building Python bindings with Vulkan enabled. Use the standalone reproduction and VulkanPartitioner entry point to confirm the issue; done means the ConvNeXt model runs without the reported runtime exception.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, pytorch
Domain
embedded-iot, machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.