facebookresearch / facebookresearch/segment-anything

c++ onnxruntime linux gpu 1.14.1 can't infer

Open
#294 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
Jupyter Notebook
Stars
54.9k
Forks
6.4k
PR merge metrics
No merged PRs in 30d

Description

when infer decoder onnx model, some times got error in session.Run, some times got 0x0 size mask output. but python version onnxruntime-gpu 1.14.1 is work fine.
error as following:
```
2023-04-26 10:54:56.695981402 [E:onnxruntime:, sequential_executor.cc:494 ExecuteKernel] Non-zero status code returned while running Resize node. Name:'/Resize_1' Status Message: upsamplebase.h:334 ScalesValidation Scale value should be greater than 0.
Non-zero status code returned while running Resize node. Name:'/Resize_1' Status Message: upsamplebase.h:334 ScalesValidation Scale value should be greater than 0.
```
or 0x0 size mask:
```
masks out shape: 1 4 0 0
iou_predictions out shape: 1 4
low_res_masks out shape: 1 4 256 256
```
onnx models:
```
wget https://huggingface.co/visheratin/segment-anything-vit-b/resolve/main/encoder-quant.onnx
wget https://huggingface.co/visheratin/segment-anything-vit-b/resolve/main/decoder-quant.onnx
```
infer code:
```c++

std::vector>>
Model::execute(std::vector> input_datas) {
auto g_values = ((GlobalValues *) globel_values_);
std::vector input_tensors;
std::vector input_names_cstr;
for (int i = 0; i < g_values->input_names.size(); i++) {
input_names_cstr.emplace_back(g_values->input_names[i].c_str());
// size_t input_tensor_size = utils::vectorProduct(g_values->input_tensor_shape[i]);
std::vector input_tensor_values(input_datas[i].data(), input_datas[i].data() + input_datas[i].size());
auto new_shape = g_values->input_tensor_shape[i];
int negind=-1;
size_t mulsize=1;
for (int j=0;jinput_tensor_shape[i].size();j++){
if(g_values->input_tensor_shape[i][j]==-1){
negind=j;
}else{
mulsize*=g_values->input_tensor_shape[i][j];
}
}
if(negind>0){
new_shape[negind]=input_datas[i].size()/mulsize;
}

Ort::MemoryInfo memory_info = Ort::MemoryInfo::CreateCpu(
OrtAllocatorType::OrtArenaAllocator, OrtMemType::OrtMemTypeDefault);
input_tensors.emplace_back(Ort::Value::CreateTensor(
memory_info, input_tensor_values.data(), input_datas[i].size(),
new_shape.data(), g_values->input_tensor_shape[i].size()
));
}

std::vector output_names_cstr;
for (int i = 0; i < g_values->output_names.size(); i++) {
output_names_cstr.emplace_back(g_values->output_names[i].c_str());
}
// uint64_t startPostTime2 = Perception::utils::getMonotonicTimeMs();
std::vector output_tensors = g_values->session.Run(Ort::RunOptions{nullptr},
input_names_cstr.data(),
input_tensors.data(),
input_tensors.size(),
output_names_cstr.data(),
output_names_cstr.size());
// std::cerr << "infer inside Process elapsed: " << Perception::utils::getMonotonicTimeMs() - startPostTime2
// << "(ms)" << std::endl;

std::vector>> out_tensors;
g_values->output_tensor_shape.clear();
for (int i = 0; i < g_values->output_names.size(); i++) {
std::vector output_shape = output_tensors[i].GetTensorTypeAndShapeInfo().GetShape();
size_t count = output_tensors[i].GetTensorTypeAndShapeInfo().GetElementCount();
auto *rawOutput = output_tensors[i].GetTensorData();
std::shared_ptr> output = std::make_shared>(rawOutput,
rawOutput + count);
out_tensors.emplace_back(output);
g_values->output_tensor_shape.emplace_back(output_shape);
std::cout << "out shape: ";
for (auto shape: output_shape)
std::cout << shape << "\t";
std::cout << std::endl;
}
return out_tensors;
}
```

Contributor guide

Open the contributing guide

Research direction

Start at Model::execute, especially tensor-shape construction and the session.Run call, using the supplied encoder-quant.onnx and decoder-quant.onnx models. Compare the C++ inference results with the working Python onnxruntime-gpu 1.14.1 run; done means the Resize error is gone and masks no longer have 0x0 dimensions.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
computer-vision, machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.