Build tensorrt engine failed with error `Could not find any implementation for node` when refit is enabled.

Description

We are trying to deploy diffusion model to tensorrt, and when the refit option is enabled, the deployment process will fail with this error:

[E] 10: Could not find any implementation for node {ForeignNode[input_blocks.1.1.transformer_blocks.0.norm1.weight + (Unnamed Layer* 170) [Shuffle].../input_blocks.1/input_blocks.1.1/Reshape_2 + /input_blocks.1/input_blocks.1.1/Transpose_1 + /input_blocks.1/input_blocks.1.1/Reshape_3]}.
[E] 10: [optimizer.cpp::computeCosts::3873] Error Code 10: Internal Error (Could not find any implementation for node {ForeignNode[input_blocks.1.1.transformer_blocks.0.norm1.weight + (Unnamed Layer* 170) [Shuffle].../input_blocks.1/input_blocks.1.1/Reshape_2 + /input_blocks.1/input_blocks.1.1/Transpose_1 + /input_blocks.1/input_blocks.1.1/Reshape_3]}.)

If refit is not enabled, the deployment can succeed.

The full log is below:

Building TensorRT engine for ./lora_trt/6424_8029/unet.onnx: ./lora_trt/6424_8029/unet_opt.plan
[W] 'colored' module is not installed, will not use colors when logging. To enable colors, please install the 'colored' module: python3 -m pip install colored
[W] onnx2trt_utils.cpp:374: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
[I]     Configuring with profiles: [Profile().add('input0', min=[2, 4, 64, 64], opt=[2, 4, 64, 64], max=[2, 4, 140, 140]).add('input1', min=[2], opt=[2], max=[2]).add('input2', min=[2, 77, 768], opt=[2, 77, 768], max=[2, 308, 768])]
[I] Building engine with configuration:
    Flags                  | [FP16, REFIT]
    Engine Capability      | EngineCapability.DEFAULT
    Memory Pools           | [WORKSPACE: 81920.00 MiB, TACTIC_DRAM: 81251.19 MiB]
    Tactic Sources         | [CUBLAS, CUBLAS_LT, CUDNN, EDGE_MASK_CONVOLUTIONS, JIT_CONVOLUTIONS]
    Profiling Verbosity    | ProfilingVerbosity.DETAILED
    Preview Features       | [FASTER_DYNAMIC_SHAPES_0805]
[W] TensorRT was linked against cuDNN 8.8.0 but loaded cuDNN 8.5.0
[E] 2: [virtualMemoryBuffer.cpp::resizePhysical::145] Error Code 2: OutOfMemory (no further information)
[E] 2: [virtualMemoryBuffer.cpp::resizePhysical::145] Error Code 2: OutOfMemory (no further information)
[W] Requested amount of GPU memory (51863468267 bytes) could not be allocated. There may not be enough free memory for allocation to succeed.
[E] 2: [virtualMemoryBuffer.cpp::resizePhysical::145] Error Code 2: OutOfMemory (no further information)
[E] 2: [virtualMemoryBuffer.cpp::resizePhysical::145] Error Code 2: OutOfMemory (no further information)
[W] Requested amount of GPU memory (25582702551 bytes) could not be allocated. There may not be enough free memory for allocation to succeed.
[E] 2: [virtualMemoryBuffer.cpp::resizePhysical::145] Error Code 2: OutOfMemory (no further information)
[E] 2: [virtualMemoryBuffer.cpp::resizePhysical::145] Error Code 2: OutOfMemory (no further information)
[E] 10: Could not find any implementation for node {ForeignNode[input_blocks.1.1.transformer_blocks.0.norm1.weight + (Unnamed Layer* 170) [Shuffle].../input_blocks.1/input_blocks.1.1/Reshape_2 + /input_blocks.1/input_blocks.1.1/Transpose_1 + /input_blocks.1/input_blocks.1.1/Reshape_3]}.
[E] 10: [optimizer.cpp::computeCosts::3873] Error Code 10: Internal Error (Could not find any implementation for node {ForeignNode[input_blocks.1.1.transformer_blocks.0.norm1.weight + (Unnamed Layer* 170) [Shuffle].../input_blocks.1/input_blocks.1.1/Reshape_2 + /input_blocks.1/input_blocks.1.1/Transpose_1 + /input_blocks.1/input_blocks.1.1/Reshape_3]}.)
[!] Invalid Engine. Please ensure the engine was built correctly

Environment

TensorRT Version: 8.6.0
NVIDIA GPU: NVIDIA A100
NVIDIA Driver Version: 470.129.06
CUDA Version: 12.0
CUDNN Version:
Operating System:
Python Version (if applicable):
Tensorflow Version (if applicable):
PyTorch Version (if applicable):
Baremetal or Container (if so, version):

Relevant Files

Steps To Reproduce

Start with the reported unet.onnx build that produces unet_opt.plan, using the TensorRT 8.6 configuration shown in the log. Compare builds with refit enabled and disabled, and investigate the reported GPU out-of-memory messages alongside the implementation error. Done means a reproducible diagnosis or confirmed fix for the refit build failure.