pytorch / pytorch/TensorRT

❓ [Question] How to specific aten operators must be run by LibTorch in C++?

Open
#2,830 7 comments 0 reactions 2 assignees View on GitHub

@bowang007 is already working on this.

Since May 14, 2024.

question story: Documentation & Examples
Dominant language
Python
Stars
3k
Forks
410
Avg merge
3d 18h
Merged PRs (30d)
78

Description

❓ Question

When I compile the SwinTransformer model using Torch-TensorRT, an error appears:

terminate called after throwing an instance of 'c10::Error'
  what():  0 INTERNAL ASSERT FAILED at "../torch/csrc/jit/ir/alias_analysis.cpp":615, please report a bug to PyTorch. We don't have an op for aten::floor_divide but it isn't a special case.  Argument types: int, int, 

Candidates:
        aten::floor_divide(Tensor self, Tensor other) -> Tensor
        aten::floor_divide.Scalar(Tensor self, Scalar other) -> Tensor
        aten::floor_divide.out(Tensor self, Tensor other, *, Tensor(a!) out) -> Tensor(a!)
        aten::floor_divide.Scalar_out(Tensor self, Scalar other, *, Tensor(a!) out) -> Tensor(a!)

I checked out this link, This error is because torch-trt dont support % op.

Fine, I can select to run floor_divide using LibTorch.

torchtrt::ts::CompileSpec compile_settings({ input });
compile_settings.enabled_precisions.insert(build_type);
compile_settings.workspace_size = _1_GB;
compile_settings.truncate_long_and_double = true;
compile_settings.num_avg_timing_iters = 1;
compile_settings.torch_executed_ops.push_back("aten::floor_divide");  // here
torchtrt::ts::compile(model, compile_settings)

It's strange that the setting does not take effect. This error still persists.

What can I do about this mistake?

Furthermore, How to specific aten operators must be run by LibTorch in C++?

Environment

Build information about Torch-TensorRT can be found by turning on debug messages

  • PyTorch Version (e.g., 1.0):2.2.1
  • CPU Architecture:x86
  • OS (e.g., Linux):ubuntu22.04
  • How you installed PyTorch (conda, pip, libtorch, source):
  • Build command you used (if compiling from source):
  • Are you using local sources or building from archives:
  • Python version:
  • CUDA version:12.2
  • GPU models and configuration:
  • Any other relevant information:

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.