microsoft / microsoft/onnxruntime
[Feature Request] Add --use_external_data_format to onnxruntime.tools.make_dynamic_shape_fixed
- Dominant language
- C++
- Stars
- 21.9k
- Forks
- 4.2k
- Avg merge
- 4d 11h
- Merged PRs (30d)
- 184
Description
### Describe the feature request
add the support of --use_external_data_format to onnxruntime.tools.make_dynamic_shape_fixed to support the >2GB model size
### Describe scenario use case
I have an onnx model with dynamic shape > 2GB size and want to make the dynamic shape fixed.
$ python -m onnxruntime.tools.make_dynamic_shape_fixed --input_name input_ids --input_shape 1,2048 --input_name attention_mask --input_shape 1,2048 model.onnx model.fixed.onnx
Traceback (most recent call last):
File "/home/aluo/miniconda3/envs/optimum/lib/python3.9/runpy.py", line 197, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/home/aluo/miniconda3/envs/optimum/lib/python3.9/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "/home/aluo/miniconda3/envs/optimum/lib/python3.9/site-packages/onnxruntime/tools/make_dynamic_shape_fixed.py", line 72, in
make_dynamic_shape_fixed_helper()
File "/home/aluo/miniconda3/envs/optimum/lib/python3.9/site-packages/onnxruntime/tools/make_dynamic_shape_fixed.py", line 66, in make_dynamic_shape_fixed_helper
fix_output_shapes(model)
File "/home/aluo/miniconda3/envs/optimum/lib/python3.9/site-packages/onnxruntime/tools/onnx_model_utils.py", line 224, in fix_output_shapes
m2 = onnx.shape_inference.infer_shapes(model)
File "/home/aluo/miniconda3/envs/optimum/lib/python3.9/site-packages/onnx/shape_inference.py", line 42, in infer_shapes
model_str = model if isinstance(model, bytes) else model.SerializeToString()
ValueError: Message onnx.ModelProto exceeds maximum protobuf size of 2GB: 5675181582
Contributor guide
Research direction
Start with onnxruntime.tools.make_dynamic_shape_fixed and the failing path through onnx_model_utils.py, especially fix_output_shapes and ONNX shape inference. Reproduce the command with a model larger than 2GB, then verify that the requested --use_external_data_format option allows dynamic shapes to be fixed without the protobuf size failure.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- cli, machine-learning
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 38/100