Internal Error (Assertion !n->candidateRequirements.empty() failed. no supported formats)
@rajeevsrao is already working on this.
Since May 15, 2023.
- Dominant language
- C++
- Stars
- 13.4k
- Forks
- 2.4k
- Avg merge
- 5d 3h
- Merged PRs (30d)
- 2
Description
Description
I am compiling an UNet model (from demo/Diffusion) from ONNX to TensorRT with plugins and int8 calibration (code from here)
During compilation, I am getting The following error:
[E] 2: [optimizer.cpp::getFormatRequirements::3015] Error Code 2: Internal Error (Assertion !n->candidateRequirements.empty() failed. no supported formats)
[E] 2: [builder.cpp::buildSerializedNetwork::738] Error Code 2: Internal Error (Assertion engine != nullptr failed. )
I use the demo from release/8.5, since the release/8.6 cannot work for me (this might be caused by the older version of my CUDA)
I can run the demo without int8 calibration normally, but when I use the following code in demo/Diffusion/utilities.py:56 to build the engine with the int8 calibration, the above error would occur.
int8_calibrator = Calibrator(data_loader=self._random_data_generator())
engine = engine_from_network(
network_from_onnx_path(onnx_path),
config=CreateConfig(
fp16=fp16, int8=False, profiles=[p],
calibrator=int8_calibrator,
preview_features=preview_features
)
)
save_engine(engine, path=self.engine_path)
def _random_data_generator(self):
for _ in range(1000):
sample = np.random.random(size=[2,4,64,64]).astype("float16")
timestep = np.random.random(size=[1]).astype("float16")
encoder_hidden_states = np.random.random(size=[2,77,768]).astype("float16")
data = {
"sample": sample,
"timestep": timestep,
"encoder_hidden_states": encoder_hidden_states
}
yield data
Besides the above part, I didn't modify other code in this demo.
Some other information might be help:
- I successfully use the
trtexectools to convert the ONNX to TRT engine with int8 mode. Buttrtexectools cannot support to use my own calibrator. - I have suspected the above error might come from the unsupported plugins for the Stable Diffusion model during the int8 calibration. But after I disable the plugin and only convert the original ONNX model before optimizing, I got the exactly same error messages as above.
Environment
TensorRT Version: 8.5.3.1
NVIDIA GPU: RTX2080/RTX3090
NVIDIA Driver Version: 510.54
CUDA Version: 11.6
CUDNN Version: 8.6.0
Operating System:
Python Version (if applicable): 3.8.10
Tensorflow Version (if applicable):
PyTorch Version (if applicable): 1.12.0+cu116
Baremetal or Container (if so, version): nvcr.io/nvidia/tensorrt:22.10-py3
Relevant Files
Model link:
Steps To Reproduce
Commands or scripts:
Have you tried the latest release?:
Can this model run on other frameworks? For example run ONNX model with ONNXRuntime (polygraphy run <model.onnx> --onnxrt):
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.