facebookresearch / facebookresearch/segment-anything
c++ onnxruntime linux gpu 1.14.1 can't infer
- Dominant language
- Jupyter Notebook
- Stars
- 54.9k
- Forks
- 6.4k
- PR merge metrics
- No merged PRs in 30d
Description
when infer decoder onnx model, some times got error in session.Run, some times got 0x0 size mask output. but python version onnxruntime-gpu 1.14.1 is work fine.
error as following:
```
2023-04-26 10:54:56.695981402 [E:onnxruntime:, sequential_executor.cc:494 ExecuteKernel] Non-zero status code returned while running Resize node. Name:'/Resize_1' Status Message: upsamplebase.h:334 ScalesValidation Scale value should be greater than 0.
Non-zero status code returned while running Resize node. Name:'/Resize_1' Status Message: upsamplebase.h:334 ScalesValidation Scale value should be greater than 0.
```
or 0x0 size mask:
```
masks out shape: 1 4 0 0
iou_predictions out shape: 1 4
low_res_masks out shape: 1 4 256 256
```
onnx models:
```
wget https://huggingface.co/visheratin/segment-anything-vit-b/resolve/main/encoder-quant.onnx
wget https://huggingface.co/visheratin/segment-anything-vit-b/resolve/main/decoder-quant.onnx
```
infer code:
```c++
std::vector>>
Model::execute(std::vector> input_datas) {
auto g_values = ((GlobalValues *) globel_values_);
std::vector input_tensors;
std::vector input_names_cstr;
for (int i = 0; i < g_values->input_names.size(); i++) {
input_names_cstr.emplace_back(g_values->input_names[i].c_str());
// size_t input_tensor_size = utils::vectorProduct(g_values->input_tensor_shape[i]);
std::vector input_tensor_values(input_datas[i].data(), input_datas[i].data() + input_datas[i].size());
auto new_shape = g_values->input_tensor_shape[i];
int negind=-1;
size_t mulsize=1;
for (int j=0;jinput_tensor_shape[i].size();j++){
if(g_values->input_tensor_shape[i][j]==-1){
negind=j;
}else{
mulsize*=g_values->input_tensor_shape[i][j];
}
}
if(negind>0){
new_shape[negind]=input_datas[i].size()/mulsize;
}
Ort::MemoryInfo memory_info = Ort::MemoryInfo::CreateCpu(
OrtAllocatorType::OrtArenaAllocator, OrtMemType::OrtMemTypeDefault);
input_tensors.emplace_back(Ort::Value::CreateTensor(
memory_info, input_tensor_values.data(), input_datas[i].size(),
new_shape.data(), g_values->input_tensor_shape[i].size()
));
}
std::vector output_names_cstr;
for (int i = 0; i < g_values->output_names.size(); i++) {
output_names_cstr.emplace_back(g_values->output_names[i].c_str());
}
// uint64_t startPostTime2 = Perception::utils::getMonotonicTimeMs();
std::vector output_tensors = g_values->session.Run(Ort::RunOptions{nullptr},
input_names_cstr.data(),
input_tensors.data(),
input_tensors.size(),
output_names_cstr.data(),
output_names_cstr.size());
// std::cerr << "infer inside Process elapsed: " << Perception::utils::getMonotonicTimeMs() - startPostTime2
// << "(ms)" << std::endl;
std::vector>> out_tensors;
g_values->output_tensor_shape.clear();
for (int i = 0; i < g_values->output_names.size(); i++) {
std::vector output_shape = output_tensors[i].GetTensorTypeAndShapeInfo().GetShape();
size_t count = output_tensors[i].GetTensorTypeAndShapeInfo().GetElementCount();
auto *rawOutput = output_tensors[i].GetTensorData();
std::shared_ptr> output = std::make_shared>(rawOutput,
rawOutput + count);
out_tensors.emplace_back(output);
g_values->output_tensor_shape.emplace_back(output_shape);
std::cout << "out shape: ";
for (auto shape: output_shape)
std::cout << shape << "\t";
std::cout << std::endl;
}
return out_tensors;
}
```
Contributor guide
Research direction
Start at Model::execute, especially tensor-shape construction and the session.Run call, using the supplied encoder-quant.onnx and decoder-quant.onnx models. Compare the C++ inference results with the working Python onnxruntime-gpu 1.14.1 run; done means the Resize error is gone and masks no longer have 0x0 dimensions.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- computer-vision, machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100