NVIDIA / NVIDIA/TensorRT

Internal Error (Assertion !n->candidateRequirements.empty() failed. no supported formats)

Open
#2,966 6 comments 0 reactions 2 assignees View on GitHub

@rajeevsrao is already working on this.

Since May 15, 2023.

triaged
Dominant language
C++
Stars
13.4k
Forks
2.4k
Avg merge
5d 3h
Merged PRs (30d)
2

Description

Description

I am compiling an UNet model (from demo/Diffusion) from ONNX to TensorRT with plugins and int8 calibration (code from here)
During compilation, I am getting The following error:

[E] 2: [optimizer.cpp::getFormatRequirements::3015] Error Code 2: Internal Error (Assertion !n->candidateRequirements.empty() failed. no supported formats)
[E] 2: [builder.cpp::buildSerializedNetwork::738] Error Code 2: Internal Error (Assertion engine != nullptr failed. )

I use the demo from release/8.5, since the release/8.6 cannot work for me (this might be caused by the older version of my CUDA)

I can run the demo without int8 calibration normally, but when I use the following code in demo/Diffusion/utilities.py:56 to build the engine with the int8 calibration, the above error would occur.

int8_calibrator = Calibrator(data_loader=self._random_data_generator())
engine = engine_from_network(
        network_from_onnx_path(onnx_path),
        config=CreateConfig(
            fp16=fp16, int8=False, profiles=[p],
            calibrator=int8_calibrator,
            preview_features=preview_features
        )
)
save_engine(engine, path=self.engine_path)


def _random_data_generator(self):
    for _ in range(1000):
        sample = np.random.random(size=[2,4,64,64]).astype("float16")
        timestep = np.random.random(size=[1]).astype("float16")
        encoder_hidden_states = np.random.random(size=[2,77,768]).astype("float16")
        data = {
            "sample": sample,
            "timestep": timestep,
            "encoder_hidden_states": encoder_hidden_states
         }
        yield data

Besides the above part, I didn't modify other code in this demo.

Some other information might be help:

  1. I successfully use the trtexec tools to convert the ONNX to TRT engine with int8 mode. But trtexec tools cannot support to use my own calibrator.
  2. I have suspected the above error might come from the unsupported plugins for the Stable Diffusion model during the int8 calibration. But after I disable the plugin and only convert the original ONNX model before optimizing, I got the exactly same error messages as above.

Environment

TensorRT Version: 8.5.3.1

NVIDIA GPU: RTX2080/RTX3090

NVIDIA Driver Version: 510.54

CUDA Version: 11.6

CUDNN Version: 8.6.0

Operating System:

Python Version (if applicable): 3.8.10

Tensorflow Version (if applicable):

PyTorch Version (if applicable): 1.12.0+cu116

Baremetal or Container (if so, version): nvcr.io/nvidia/tensorrt:22.10-py3

Relevant Files

Model link:

Steps To Reproduce

Commands or scripts:

Have you tried the latest release?:

Can this model run on other frameworks? For example run ONNX model with ONNXRuntime (polygraphy run <model.onnx> --onnxrt):

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.