microsoft / microsoft/onnxruntime-inference-examples

Help!:Bert-GLUE_OnnxRuntime_quantization.ipynb AssertionError:

Open
#57 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug
Dominant language
C++
Stars
1.7k
Forks
414
Avg merge
1d 6h
Merged PRs (30d)
14

Description

when I run the Bert-GLUE_OnnxRuntime_quantization.ipynb in google colab,I got the follow error:

Warning: onnxruntime_tools is deprecated. Use onnxruntime or onnxruntime-gpu instead. For more information, see https://github.com/microsoft/onnxruntime/blob/master/onnxruntime/python/tools/transformers/README.md.


AssertionError Traceback (most recent call last)
in ()
40 num_heads=12,
41 hidden_size=768,
---> 42 optimization_options=opt_options)
43 opt_model.save_model_to_file('bert.opt.onnx')

8 frames
/usr/local/lib/python3.7/dist-packages/onnxruntime_tools/transformers/optimizer.py in optimize_model(input, model_type, num_heads, hidden_size, optimization_options, opt_level, use_gpu, only_onnxruntime)
310
311 if not only_onnxruntime:
--> 312 optimizer.optimize(optimization_options)
313
314 # Remove the temporary model.

/usr/local/lib/python3.7/dist-packages/onnxruntime_tools/transformers/onnx_model_bert.py in optimize(self, options, add_dynamic_axes)
277
278 if (options is None) or options.enable_skip_layer_norm:
--> 279 self.fuse_skip_layer_norm()
280
281 if (options is None) or options.enable_attention:

/usr/local/lib/python3.7/dist-packages/onnxruntime_tools/transformers/onnx_model_bert.py in fuse_skip_layer_norm(self)
103
104 def fuse_skip_layer_norm(self):
--> 105 fusion = FusionSkipLayerNormalization(self)
106 fusion.apply()
107

/usr/local/lib/python3.7/dist-packages/onnxruntime_tools/transformers/fusion_skiplayernorm.py in init(self, model)
19 def init(self, model: OnnxModel):
20 super().init(model, "SkipLayerNormalization", "LayerNormalization")
---> 21 self.shape_infer_helper = self.model.infer_runtime_shape({"batch_size": 4, "seq_len": 7})
22
23 def fuse(self, node, input_name_to_nodes, output_name_to_node):

/usr/local/lib/python3.7/dist-packages/onnxruntime_tools/transformers/onnx_model.py in infer_runtime_shape(self, dynamic_axis_mapping, update)
34 shape_infer_helper = self.shape_infer_helper
35
---> 36 if shape_infer_helper.infer(dynamic_axis_mapping):
37 return shape_infer_helper
38 return None

/usr/local/lib/python3.7/dist-packages/onnxruntime_tools/transformers/shape_infer_helper.py in infer(self, dynamic_axis_mapping)
33 self.preprocess(self.model)
34 while self.run_:
---> 35 self.all_shapes_inferred_ = self.infer_impl()
36
37 self.inferred
= True

/usr/local/lib/python3.7/dist-packages/onnxruntime_tools/transformers/../symbolic_shape_infer.py in _infer_impl(self, start_sympy_data)
1301 in_dims = [s[len(s) - out_rank + d] for s in in_shapes if len(s) + d >= out_rank]
1302 if len(in_dims) > 1:
-> 1303 self._check_merged_dims(in_dims, allow_broadcast=True)
1304
1305 for i_o in range(len(node.output)):

/usr/local/lib/python3.7/dist-packages/onnxruntime_tools/transformers/../symbolic_shape_infer.py in _check_merged_dims(self, dims, allow_broadcast)
527 dims = [d for d in dims if not (is_literal(d) and int(d) <= 1)]
528 if not all([d == dims[0] for d in dims]):
--> 529 self._add_suggested_merge(dims, apply=True)
530
531 def _compute_matmul_shape(self, node, output_dtype=None):

/usr/local/lib/python3.7/dist-packages/onnxruntime_tools/transformers/../symbolic_shape_infer.py in add_suggested_merge(self, symbols, apply)
156
157 def add_suggested_merge(self, symbols, apply=False):
--> 158 assert all([(type(s) == str and s in self.symbolic_dims
) or is_literal(s) for s in symbols])
159 symbols = set(symbols)
160 for k, v in self.suggested_merge
.items():

AssertionError:

any one know how to fix this error?

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with Bert-GLUE_OnnxRuntime_quantization.ipynb and reproduce the optimization step in Google Colab, checking the installed Python and ONNX Runtime-related package versions. Trace the AssertionError from the notebook's optimize_model call; done means the notebook completes model optimization and quantization without the assertion.

Written by the indexing model from the issue text.

Assessment

Tech stack
jupyter-notebook, python
Domain
machine-learning
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.