microsoft / microsoft/onnxruntime-inference-examples
Help!:Bert-GLUE_OnnxRuntime_quantization.ipynb AssertionError:
Nobody has claimed this yet.
- Dominant language
- C++
- Stars
- 1.7k
- Forks
- 414
- Avg merge
- 1d 6h
- Merged PRs (30d)
- 14
Description
when I run the Bert-GLUE_OnnxRuntime_quantization.ipynb in google colab,I got the follow error:
Warning: onnxruntime_tools is deprecated. Use onnxruntime or onnxruntime-gpu instead. For more information, see https://github.com/microsoft/onnxruntime/blob/master/onnxruntime/python/tools/transformers/README.md.
AssertionError Traceback (most recent call last)
in ()
40 num_heads=12,
41 hidden_size=768,
---> 42 optimization_options=opt_options)
43 opt_model.save_model_to_file('bert.opt.onnx')
8 frames
/usr/local/lib/python3.7/dist-packages/onnxruntime_tools/transformers/optimizer.py in optimize_model(input, model_type, num_heads, hidden_size, optimization_options, opt_level, use_gpu, only_onnxruntime)
310
311 if not only_onnxruntime:
--> 312 optimizer.optimize(optimization_options)
313
314 # Remove the temporary model.
/usr/local/lib/python3.7/dist-packages/onnxruntime_tools/transformers/onnx_model_bert.py in optimize(self, options, add_dynamic_axes)
277
278 if (options is None) or options.enable_skip_layer_norm:
--> 279 self.fuse_skip_layer_norm()
280
281 if (options is None) or options.enable_attention:
/usr/local/lib/python3.7/dist-packages/onnxruntime_tools/transformers/onnx_model_bert.py in fuse_skip_layer_norm(self)
103
104 def fuse_skip_layer_norm(self):
--> 105 fusion = FusionSkipLayerNormalization(self)
106 fusion.apply()
107
/usr/local/lib/python3.7/dist-packages/onnxruntime_tools/transformers/fusion_skiplayernorm.py in init(self, model)
19 def init(self, model: OnnxModel):
20 super().init(model, "SkipLayerNormalization", "LayerNormalization")
---> 21 self.shape_infer_helper = self.model.infer_runtime_shape({"batch_size": 4, "seq_len": 7})
22
23 def fuse(self, node, input_name_to_nodes, output_name_to_node):
/usr/local/lib/python3.7/dist-packages/onnxruntime_tools/transformers/onnx_model.py in infer_runtime_shape(self, dynamic_axis_mapping, update)
34 shape_infer_helper = self.shape_infer_helper
35
---> 36 if shape_infer_helper.infer(dynamic_axis_mapping):
37 return shape_infer_helper
38 return None
/usr/local/lib/python3.7/dist-packages/onnxruntime_tools/transformers/shape_infer_helper.py in infer(self, dynamic_axis_mapping)
33 self.preprocess(self.model)
34 while self.run_:
---> 35 self.all_shapes_inferred_ = self.infer_impl()
36
37 self.inferred = True
/usr/local/lib/python3.7/dist-packages/onnxruntime_tools/transformers/../symbolic_shape_infer.py in _infer_impl(self, start_sympy_data)
1301 in_dims = [s[len(s) - out_rank + d] for s in in_shapes if len(s) + d >= out_rank]
1302 if len(in_dims) > 1:
-> 1303 self._check_merged_dims(in_dims, allow_broadcast=True)
1304
1305 for i_o in range(len(node.output)):
/usr/local/lib/python3.7/dist-packages/onnxruntime_tools/transformers/../symbolic_shape_infer.py in _check_merged_dims(self, dims, allow_broadcast)
527 dims = [d for d in dims if not (is_literal(d) and int(d) <= 1)]
528 if not all([d == dims[0] for d in dims]):
--> 529 self._add_suggested_merge(dims, apply=True)
530
531 def _compute_matmul_shape(self, node, output_dtype=None):
/usr/local/lib/python3.7/dist-packages/onnxruntime_tools/transformers/../symbolic_shape_infer.py in add_suggested_merge(self, symbols, apply)
156
157 def add_suggested_merge(self, symbols, apply=False):
--> 158 assert all([(type(s) == str and s in self.symbolic_dims) or is_literal(s) for s in symbols])
159 symbols = set(symbols)
160 for k, v in self.suggested_merge.items():
AssertionError:
any one know how to fix this error?
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with Bert-GLUE_OnnxRuntime_quantization.ipynb and reproduce the optimization step in Google Colab, checking the installed Python and ONNX Runtime-related package versions. Trace the AssertionError from the notebook's optimize_model call; done means the notebook completes model optimization and quantization without the assertion.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- jupyter-notebook, python
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100