NVIDIA / NVIDIA/TensorRT

TensorRT Plugin gets incorrect input data when integrated into full model, but works fine in isolation

Open
#4,440 1 comment 0 reactions 1 assignee View on GitHub

@venkywonka is already working on this.

Since May 19, 2025.

Module:Plugins triaged
Dominant language
C++
Stars
13.4k
Forks
2.4k
Avg merge
5d 3h
Merged PRs (30d)
2

Description

I am converting a custom PyTorch plugin (CorrSamplerPlugin) into a TensorRT plugin. When testing the plugin independently, the results are correct and consistent with PyTorch. However, after integrating the plugin into the full model and exporting it to a TensorRT engine, the plugin receives incorrect input data during inference, leading to wrong outputs.

Current status:
The plugin is verified independently with correct input/output compared to PyTorch;

After full model integration, the volume input received by the plugin differs significantly from the PyTorch version;

The calling order of plugin layers seems wrong (e.g., CorrSampler_0, CorrSampler_3, CorrSampler_2, CorrSampler_1, CorrSampler_4);

During inference, volume input values show large differences (even signs are flipped), while coord values only match in early iterations and then diverge;

Confirmed:

No memory errors inside the plugin;

Input shapes and binding names match one-by-one;

FP16 and INT8 are disabled — FP32 is used;

supportsFormatCombination enforces kFLOAT and kLINEAR;

I listed top 10 shapes and values of my test datas on the c++ tensorrt inference:

Image

Here I alse listed top 10 shapes and values of my test datas on the pytorch:

Image

This is my op:

Image

Image

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.