onnx / onnx/models

Dynamic batch sizes are not supported in tiny-yolov3 model

Open
#678 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug
Dominant language
Jupyter Notebook
Stars
9.8k
Forks
1.6k
PR merge metrics
No merged PRs in 30d

Description

Bug Report

Which model does this pertain to?

Tiny-yolov3 folder

Describe the bug

Model does not support batch sizes > 1.

error :

----------- testing start ----------
input data name : input_1, shape = (2, 3, 416, 416), type = <class 'numpy.ndarray'>
input data name : image_shape, shape = (2, 2), type = <class 'numpy.ndarray'>
2025-03-19 23:19:18.958568678 [E:onnxruntime:, sequential_executor.cc:516 ExecuteKernel] Non-zero status code returned while running Squeeze node. Name:'TFNodes/yolo_evaluation_layer_1/Squeeze' Status Message: /onnxruntime_src/onnxruntime/core/providers/cpu/tensor/squeeze.h:52 static onnxruntime::TensorShapeVector onnxruntime::SqueezeBase::ComputeOutputShape(const onnxruntime::TensorShape&, const onnxruntime::TensorShapeVector&) input_shape[i] == 1 was false. Dimension of input 0 must be 1 instead of 2. shape={2,2}

Reproduction instructions

Run model inference on any batch size > 1.

System Information

OS Platform and Distribution (e.g. Linux Ubuntu 16.04):
ONNX version (e.g. 1.6):
Backend/Runtime version (e.g. ONNX Runtime 1.1, PyTorch 1.2):

Provide a code snippet to reproduce your errors.

import numpy as np 

    def test_call(self, ):
        
        for batch_size in [1, 2]:

            image = np.random.rand(batch_size, 3, 416, 416,).astype(np.float32)
            image_shape = np.array([[416, 416]], dtype=np.float32).reshape(1, 2)
            if batch_size != 1:
                image_shape = np.vstack([ image_shape for _ in range(batch_size)])
                
            inputs = {
                self.input_names[0]: image,
                self.input_names[1]: image_shape
            }
            print(f"----------- testing start ----------")
            for key, value in inputs.items():
                print(f"input data name : {key}, shape = {value.shape}, type = {type(value)}")
            
            # any forward wrapper that takes in the input and returns the output dict.
            output_dict = self.forward(inputs)
            

            for key, value in output_dict.items():
                print(f"output data name : {key}, shape = {value.shape}, type = {type(value)}")
...
Notes
  1. runs correct with batch size 1.
  2. fails on any batch size > 1

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in the validated/vision/object_detection_segmentation/tiny-yolov3 folder and reproduce inference with the provided batch-size loop, focusing on the Squeeze node error for image_shape with shape (2,2). Done means the tiny-yolov3 model runs successfully for batch size 1 and batch sizes greater than 1.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
computer-vision, machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.