microsoft / microsoft/onnxruntime

[Feature Request] Add --use_external_data_format to onnxruntime.tools.make_dynamic_shape_fixed

Open
#16,773 0 comments 0 reactions 0 assignees View on GitHub
feature request
Dominant language
C++
Stars
21.9k
Forks
4.2k
Avg merge
4d 11h
Merged PRs (30d)
184

Description

### Describe the feature request

add the support of --use_external_data_format to onnxruntime.tools.make_dynamic_shape_fixed to support the >2GB model size

### Describe scenario use case

I have an onnx model with dynamic shape > 2GB size and want to make the dynamic shape fixed.

$ python -m onnxruntime.tools.make_dynamic_shape_fixed --input_name input_ids --input_shape 1,2048 --input_name attention_mask --input_shape 1,2048 model.onnx model.fixed.onnx
Traceback (most recent call last):
File "/home/aluo/miniconda3/envs/optimum/lib/python3.9/runpy.py", line 197, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/home/aluo/miniconda3/envs/optimum/lib/python3.9/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "/home/aluo/miniconda3/envs/optimum/lib/python3.9/site-packages/onnxruntime/tools/make_dynamic_shape_fixed.py", line 72, in
make_dynamic_shape_fixed_helper()
File "/home/aluo/miniconda3/envs/optimum/lib/python3.9/site-packages/onnxruntime/tools/make_dynamic_shape_fixed.py", line 66, in make_dynamic_shape_fixed_helper
fix_output_shapes(model)
File "/home/aluo/miniconda3/envs/optimum/lib/python3.9/site-packages/onnxruntime/tools/onnx_model_utils.py", line 224, in fix_output_shapes
m2 = onnx.shape_inference.infer_shapes(model)
File "/home/aluo/miniconda3/envs/optimum/lib/python3.9/site-packages/onnx/shape_inference.py", line 42, in infer_shapes
model_str = model if isinstance(model, bytes) else model.SerializeToString()
ValueError: Message onnx.ModelProto exceeds maximum protobuf size of 2GB: 5675181582

Contributor guide

Open the contributing guide

Research direction

Start with onnxruntime.tools.make_dynamic_shape_fixed and the failing path through onnx_model_utils.py, especially fix_output_shapes and ONNX shape inference. Reproduce the command with a model larger than 2GB, then verify that the requested --use_external_data_format option allows dynamic shapes to be fixed without the protobuf size failure.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
cli, machine-learning
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.