when use IInt8EntropyCalibrator2 multi input there is some problem
Open
@zerollzeng is already working on this.
Since Mar 28, 2023.
triaged
- Dominant language
- C++
- Stars
- 13.4k
- Forks
- 2.4k
- Avg merge
- 5d 3h
- Merged PRs (30d)
- 2
Description
Description
When I use IInt8EntropyCalibrator2, there are 7 input. so I cudaMemcpy 7 input to bindings in Int8EntropyCalibrator2::getBatch
like:
CUDA_CHECK(cudaMemcpy(device_input_1_, input1, INPUT_SIZE_1_C * sizeof(float), cudaMemcpyHostToDevice));
assert(!strcmp(names[0], INPUT_1_BLOB_NAME_C));
bindings[0] = device_input_1_;
CUDA_CHECK(cudaMemcpy(device_input_2_, input2, INPUT_SIZE_2_C * sizeof(float), cudaMemcpyHostToDevice));
assert(!strcmp(names[1], INPUT_2_BLOB_NAME_C));
bindings[1] = device_input_2_;
CUDA_CHECK(cudaMemcpy(device_input_3_, input3, INPUT_SIZE_3_C * sizeof(float), cudaMemcpyHostToDevice));
assert(!strcmp(names[2], INPUT_3_BLOB_NAME_C));
bindings[2] = device_input_3_;
CUDA_CHECK(cudaMemcpy(device_input_4_, input4, INPUT_SIZE_4_C * sizeof(float), cudaMemcpyHostToDevice));
assert(!strcmp(names[3], INPUT_4_BLOB_NAME_C));
bindings[3] = device_input_4_;
CUDA_CHECK(cudaMemcpy(device_input_5_, input5, INPUT_SIZE_5_C * sizeof(float), cudaMemcpyHostToDevice));
assert(!strcmp(names[4], INPUT_5_BLOB_NAME_C));
bindings[4] = device_input_5_;
CUDA_CHECK(cudaMemcpy(device_input_6_, input6, INPUT_SIZE_6_C * sizeof(float), cudaMemcpyHostToDevice));
assert(!strcmp(names[5], INPUT_6_BLOB_NAME_C));
bindings[5] = device_input_6_;
CUDA_CHECK(cudaMemcpy(device_input_7_, input7, INPUT_SIZE_7_C * sizeof(float), cudaMemcpyHostToDevice));
assert(!strcmp(names[6], INPUT_7_BLOB_NAME_C));
bindings[6] = device_input_7_;
but it report :
03/28/2023-18:37:35] [E] [TRT] 2: [helpers.h::divUp::70] Error Code 2: Internal Error (Assertion n > 0 failed. )
[03/28/2023-18:37:35] [E] [TRT] 3: [engine.cpp::~Engine::306] Error Code 3: API Usage Error (Parameter check failed at: runtime/api/engine.cpp::~Engine::306, condition: mObjectCounter.use_count() == 1. Destroying an engine object before destroying objects it created leads to undefined behavior.
)
[03/28/2023-18:37:35] [E] [TRT] 2: [calibrator.cpp::calibrateEngine::1177] Error Code 2: Internal Error (Assertion context->executeV2(&bindings[0]) failed. )
build engine done
How to solve this problem?
One input is OK!
Environment
TensorRT Version: 8.5.10
CUDA 11.4
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.