❓ [Question] How to specific aten operators must be run by LibTorch in C++?
Open
@bowang007 is already working on this.
Since May 14, 2024.
question
story: Documentation & Examples
- Dominant language
- Python
- Stars
- 3k
- Forks
- 410
- Avg merge
- 3d 18h
- Merged PRs (30d)
- 78
Description
❓ Question
When I compile the SwinTransformer model using Torch-TensorRT, an error appears:
terminate called after throwing an instance of 'c10::Error'
what(): 0 INTERNAL ASSERT FAILED at "../torch/csrc/jit/ir/alias_analysis.cpp":615, please report a bug to PyTorch. We don't have an op for aten::floor_divide but it isn't a special case. Argument types: int, int,
Candidates:
aten::floor_divide(Tensor self, Tensor other) -> Tensor
aten::floor_divide.Scalar(Tensor self, Scalar other) -> Tensor
aten::floor_divide.out(Tensor self, Tensor other, *, Tensor(a!) out) -> Tensor(a!)
aten::floor_divide.Scalar_out(Tensor self, Scalar other, *, Tensor(a!) out) -> Tensor(a!)
I checked out this link, This error is because torch-trt dont support % op.
Fine, I can select to run floor_divide using LibTorch.
torchtrt::ts::CompileSpec compile_settings({ input });
compile_settings.enabled_precisions.insert(build_type);
compile_settings.workspace_size = _1_GB;
compile_settings.truncate_long_and_double = true;
compile_settings.num_avg_timing_iters = 1;
compile_settings.torch_executed_ops.push_back("aten::floor_divide"); // here
torchtrt::ts::compile(model, compile_settings)
It's strange that the setting does not take effect. This error still persists.
What can I do about this mistake?
Furthermore, How to specific aten operators must be run by LibTorch in C++?
Environment
Build information about Torch-TensorRT can be found by turning on debug messages
- PyTorch Version (e.g., 1.0):2.2.1
- CPU Architecture:x86
- OS (e.g., Linux):ubuntu22.04
- How you installed PyTorch (
conda,pip,libtorch, source): - Build command you used (if compiling from source):
- Are you using local sources or building from archives:
- Python version:
- CUDA version:12.2
- GPU models and configuration:
- Any other relevant information:
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.