The cumulative sum operation in TensorRT 8.6 is slower than in PyTorch.

Description

After converting with PyTorch-to-ONNX-TensorRT and performing the cumulative sum (cumsum) operation, the speed delay depending on the length of the input data is much greater than in PyTorch and ONNX.

Environment

TensorRT Version: 8.6.1

NVIDIA GPU: TITAN RTX, A5000

NVIDIA Driver Version: 522.06

CUDA Version: 11.8

CUDNN Version: 8.7.0

Operating System: Windows 11, Ubuntu 20.04
Python Version (if applicable): 3.10.9
Tensorflow Version (if applicable): -
PyTorch Version (if applicable): 2.0.0, 2.0.1
Baremetal or Container (if so, version): -

Relevant Files

Model link:
Cumsum_Test.zip

Steps To Reproduce

Commands or scripts:

unzip Cumsum_Test.zip
python Cumsum_Test.py

After that, you can get a time comparison graph file 'Cumsum_Test.png'.

Have you tried the latest release?: Yes.

Can this model run on other frameworks? For example run ONNX model with ONNXRuntime (polygraphy run <model.onnx> --onnxrt): The above sample is including pytorch, onnx and tensorrt comparison tests.