NVIDIA / NVIDIA/TensorRT

error in ms_deformable_im2col_cuda: invalid configuration argument

Open
#4,260 4 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

triaged
Dominant language
C++
Stars
13.4k
Forks
2.4k
Avg merge
5d 3h
Merged PRs (30d)
2

Description

Description

The following error occurred during the process of converting the onnx file to engine on jetson orin. For specific logs, see the attached file

The execution command is
/usr/src/tensorrt/bin/trtexec --onnx=./checkpoints/onnx/bevformer_small_epoch_24_cp2_add_cast.onnx --saveEngine=bevformer_small_epoch_24_cp2_add_cast.engine --fp16 --int8 --plugins=/home/orin/disk/BEVFormer_tensorrt/TensorRT/lib/libtensorrt_ops.so --dumpLayerInfo --profilingVerbosity=detailed --exportLayerInfo=layers.json --verbose

The error message is
‘error in ms_deformable_im2col_cuda: invalid configuration argument’

When I use tensorrt8.6, the above error does not occur. So I think it may be strongly related to tensorrt version.

Environment

TensorRT Version: 10.3

NVIDIA GPU: jetson orin

CUDA Version:12.2

Jetpack Version:12.2

Operating System: 22.04

Baremetal or Container (if so, version): nvcr.io/nvidia/l4t-jetpack:r36.4.0

Relevant Files

Model link: https://drive.google.com/file/d/1rAQWJJt5qaCkxevlAnzmCgnUYfXzCHoB/view?usp=sharing

Steps To Reproduce

  1. Pull the image https://catalog.ngc.nvidia.com/orgs/nvidia/containers/l4t-jetpack/tags
  2. The custom plugin is compiled to generate libtensorrt_ops.so. (Use the plugin in the following link
    https://github.com/DerryHub/BEVFormer_tensorrt/tree/main/TensorRT/plugin)
  3. /usr/src/tensorrt/bin/trtexec --onnx=./checkpoints/onnx/bevformer_small_epoch_24_cp2_add_cast.onnx --saveEngine=bevformer_small_epoch_24_cp2_add_cast.engine --fp16 --int8 --plugins=/home/orin/disk/BEVFormer_tensorrt/TensorRT/lib/libtensorrt_ops.so --dumpLayerInfo --profilingVerbosity=detailed --exportLayerInfo=layers.json --verbose

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Reproduce the failure with the provided trtexec command in nvcr.io/nvidia/l4t-jetpack:r36.4.0, using the BEVFormer_tensorrt TensorRT/plugin source and libtensorrt_ops.so. Compare the ms_deformable_im2col_cuda behavior under TensorRT 10.3 and 8.6, then document a confirmed cause or fix for Jetson Orin; success is a working engine build without the invalid configuration error.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp, docker
Domain
backend, machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.