onnx / onnx/optimizer

The "split" passes result in invalid models.

Open
#157 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
C++
Stars
834
Forks
109
Avg merge
6h 55m
Merged PRs (30d)
2

Description

Recently, I noticed that the split pass does not work, both in init and predict operations.
In particular, it results into invalid model parts. I would expect that each part would be a valid net, but this is not the case. I invoked them separately to determine which part plays a role to it, and both passes result in invalid nets.

For example (this is encountered in numerous models):

SSD (ONNX Hub, opset=12)

For split_init pass:
"split_init": "IndexError('list index (0) out of range')"

For split_predict pass:
"split_predict": "ValueError(\"Required inputs (['Constant_724', 'Constant_723', 'Constant_721', 'Constant_711', 'Constant_708', 'Constant_698', 'Constant_696', 'Constant_694', 'Constant_693', 'Constant_692', 'Constant_691', 'Constant_689', 'Constant_685', 'ConstantOfShape_680', 'Constant_675', 'Constant_672', 'Constant_662', 'Constant_648', 'Constant_647', 'Constant_644', 'Constant_643', 'Constant_642', 'Constant_722', 'Constant_641', 'Constant_638', 'Constant_636', 'Constant_631', 'Constant_630', 'Constant_623', 'Constant_622', 'Constant_684', 'Constant_621', 'Constant_617', 'Constant_613', 'Constant_609', 'Constant_604', 'Constant_603', 'Constant_602', 'Constant_601', 'Constant_598', 'Constant_597', 'Constant_593', 'Constant_701', 'Constant_591', 'Constant_588', 'Constant_627', 'Constant_585', 'Constant_582', 'Constant_578', 'Constant_577', 'Constant_576', 'Constant_574', 'Constant_573', 'Constant_571', 'Constant_569', 'Constant_568', 'Constant_608', 'Constant_566', 'Constant_562', 'Constant_561', 'Constant_560', 'Constant_556', 'Constant_551', 'Constant_539', 'Constant_538', 'Constant_536', 'Constant_534', 'Constant_639', 'Constant_531', 'Constant_528', 'Constant_525', 'Constant_522', 'Constant_520', 'Constant_518', 'Constant_612', 'backbone.model.layer2.0.3.bn2.running_var', 'backbone.model.layer1.4.2.bn2.running_mean', 'Constant_544', 'backbone.model.layer2.0.3.bn2.running_mean', 'backbone.model.layer1.0.weight', 'backbone.model.layer2.0.2.conv2.weight', 'backbone.model.layer2.0.2.bn1.weight', 'Constant_686', 'Constant_553', 'backbone.conf.5.weight', 'Constant_527', 'backbone.model.layer2.0.2.conv1.weight', 'Unsqueeze_412', 'backbone.model.layer2.0.4.conv1.weight', 'backbone.loc.5.weight', 'ConstantOfShape_678', 'backbone.model.layer2.0.1.bn2.running_mean', 'Constant_501', 'Constant_649', 'backbone.model.layer2.0.1.bn1.running_var', 'backbone.model.layer1.4.2.conv2.weight', 'backbone.model.layer1.5.2.bn2.running_var', 'Constant_614', 'backbone.model.layer2.0.1.bn1.weight', 'backbone.model.layer2.0.1.conv1.weight', 'backbone.model.layer1.5.0.bn2.bias', 'backbone.loc.3.bias', 'backbone.model.layer2.0.0.downsample.0.weight', 'backbone.model.layer2.0.0.bn2.running_mean', 'ConstantOfShape_682', 'backbone.model.layer2.0.2.bn2.bias', 'Constant_663', 'backbone.model.layer2.0.3.bn1.weight', 'backbone.model.layer2.0.0.bn1.bias', 'backbone.model.layer1.5.3.bn2.running_mean', 'backbone.model.layer1.5.3.conv2.weight', 'Constant_493', 'Constant_667', 'backbone.model.layer1.5.3.bn1.running_mean', 'Constant_673', 'backbone.loc.1.bias', 'backbone.model.layer1.5.3.bn1.bias', 'backbone.model.layer2.0.2.bn2.running_mean', 'Constant_665', 'backbone.model.layer1.5.3.bn1.weight', 'backbone.loc.2.bias', 'Constant_494', 'backbone.model.layer1.1.running_mean', 'Constant_541', 'backbone.model.layer2.0.2.bn1.running_mean', 'backbone.model.layer1.5.2.bn1.running_var', 'backbone.model.layer1.4.1.conv1.weight', 'backbone.additional_blocks.3.2.bias', 'backbone.model.layer2.0.0.downsample.1.running_mean', 'backbone.model.layer1.5.3.bn2.running_var', 'backbone.model.layer1.5.2.bn1.running_mean', 'Constant_523', 'backbone.model.layer1.5.1.bn1.bias', 'backbone.model.layer2.0.0.bn2.running_var', 'backbone.model.layer1.5.0.bn2.running_var', 'backbone.model.layer1.5.2.bn1.bias', 'backbone.additional_blocks.2.0.bias', 'backbone.model.layer2.0.2.bn1.bias', 'backbone.model.layer1.4.2.bn2.weight', 'Unsqueeze_445', 'backbone.model.layer1.4.1.bn1.running_var', 'backbone.model.layer2.0.0.bn1.running_mean', 'backbone.model.layer1.4.1.bn2.running_var', 'backbone.model.layer1.5.2.bn2.weight', 'backbone.conf.2.bias', 'Constant_620', 'backbone.model.layer1.5.0.downsample.1.running_var', 'backbone.model.layer1.4.2.bn1.weight', 'Constant_482', 'Constant_596', 'backbone.model.layer1.5.1.bn1.running_mean', 'Unsqueeze_444', 'Constant_670', 'backbone.model.layer1.5.3.bn2.weight', 'backbone.model.layer1.4.1.bn1.bias', 'Unsqueeze_467', 'backbone.model.layer2.0.0.bn2.weight', 'Constant_567', 'Constant_510', 'backbone.model.layer1.4.0.conv2.weight', 'backbone.model.layer1.4.0.bn2.running_mean', 'backbone.model.layer1.4.1.bn2.running_mean', 'backbone.model.layer1.5.2.bn2.bias', 'backbone.additional_blocks.2.0.weight', 'backbone.model.layer2.0.3.bn1.running_var', 'backbone.model.layer2.0.1.bn2.running_var', 'backbone.model.layer1.5.0.bn1.running_var', 'Constant_515', 'backbone.model.layer1.4.0.bn1.weight', 'backbone.model.layer1.4.0.bn2.running_var', 'backbone.model.layer1.5.3.bn2.bias', 'backbone.model.layer2.0.3.conv2.weight', 'backbone.model.layer1.5.2.conv2.weight', 'Constant_543', 'backbone.model.layer1.4.2.bn1.running_mean', 'backbone.model.layer2.0.1.bn1.running_mean', 'backbone.model.layer1.4.2.bn1.bias', 'backbone.model.layer1.5.1.bn1.running_var', 'backbone.model.layer1.1.running_var', 'backbone.model.layer2.0.5.bn1.running_var', 'Constant_480', 'Constant_607', 'backbone.model.layer1.4.1.conv2.weight', 'Constant_476', 'backbone.model.layer2.0.3.bn2.bias', 'backbone.model.layer2.0.0.conv2.weight', 'backbone.model.layer2.0.1.conv2.weight', 'backbone.model.layer2.0.1.bn1.bias', 'backbone.model.layer1.5.3.conv1.weight', 'backbone.model.layer1.4.1.bn2.bias', 'Unsqueeze_422', 'backbone.model.layer1.1.weight', 'backbone.model.layer2.0.2.bn2.weight', 'backbone.model.layer1.5.1.conv1.weight', 'backbone.model.layer1.4.0.conv1.weight', 'backbone.model.layer1.5.1.bn2.bias', 'Constant_625', 'backbone.model.layer1.1.bias', 'Constant_587', 'backbone.model.layer2.0.0.bn1.running_var', 'backbone.model.layer1.4.0.bn2.weight', 'backbone.model.layer2.0.2.bn2.running_var', 'backbone.model.layer1.4.0.bn2.bias', 'backbone.model.layer1.4.2.bn1.running_var', 'backbone.model.layer1.5.0.bn2.running_mean', 'backbone.model.layer1.5.3.bn1.running_var', 'backbone.model.layer1.5.0.downsample.1.running_mean', 'Constant_563', 'backbone.model.layer2.0.0.downsample.1.running_var', 'backbone.model.layer1.4.2.bn2.bias', 'Constant_487', 'Constant_533', 'backbone.model.layer2.0.4.bn1.weight', 'backbone.model.layer2.0.3.bn1.bias', 'backbone.model.layer1.4.1.bn1.running_mean', 'backbone.loc.0.weight', 'backbone.model.layer1.4.2.bn2.running_var', 'backbone.model.layer1.4.0.bn1.running_var', 'backbone.additional_blocks.0.0.weight', 'backbone.model.layer2.0.3.bn2.weight', 'backbone.model.layer2.0.0.bn2.bias', 'backbone.model.layer1.5.0.bn1.weight', 'backbone.model.layer1.5.2.bn1.weight', 'Constant_669', 'Constant_504', 'backbone.model.layer1.5.0.bn1.bias', 'backbone.model.layer1.5.0.bn1.running_mean', 'backbone.conf.3.bias', 'Constant_558', 'backbone.additional_blocks.1.0.weight', 'Constant_438', 'backbone.model.layer1.5.1.bn2.running_mean', 'backbone.model.layer1.5.0.conv2.weight', 'backbone.model.layer1.4.1.bn1.weight', 'backbone.model.layer1.4.1.bn2.weight', 'Unsqueeze_356', 'backbone.model.layer1.5.0.bn2.weight', 'Constant_474', 'backbone.model.layer1.5.1.conv2.weight', 'backbone.model.layer1.5.0.downsample.0.weight', 'backbone.model.layer2.0.5.conv2.weight', 'Constant_579', 'backbone.model.layer2.0.0.bn1.weight', 'backbone.model.layer1.5.0.downsample.1.bias', 'backbone.model.layer2.0.5.bn2.running_mean', 'backbone.model.layer1.5.0.conv1.weight', 'backbone.model.layer1.4.0.bn1.bias', 'Constant_526', 'backbone.model.layer2.0.5.bn1.weight', 'Constant_731', 'Unsqueeze_345', 'backbone.model.layer1.5.0.downsample.1.weight', 'backbone.model.layer1.5.1.bn2.weight', 'Constant_506', 'backbone.model.layer2.0.4.bn1.running_mean', 'Constant_632', 'Unsqueeze_346', 'backbone.model.layer2.0.4.bn1.running_var', 'backbone.model.layer2.0.0.downsample.1.weight', 'backbone.model.layer2.0.4.bn2.weight', 'backbone.model.layer2.0.4.bn2.bias', 'Constant_590', 'backbone.model.layer2.0.4.conv2.weight', 'backbone.model.layer2.0.4.bn2.running_mean', 'backbone.model.layer2.0.3.conv1.weight', 'backbone.model.layer1.5.1.bn2.running_var', 'backbone.model.layer1.5.2.conv1.weight', 'backbone.model.layer2.0.4.bn2.running_var', 'backbone.model.layer2.0.5.bn1.bias', 'Constant_552', 'backbone.model.layer2.0.5.bn1.running_mean', 'backbone.model.layer2.0.5.bn2.bias', 'backbone.model.layer2.0.5.bn2.running_var', 'backbone.additional_blocks.0.0.bias', 'backbone.additional_blocks.0.2.weight', 'Constant_532', 'backbone.additional_blocks.0.2.bias', 'backbone.loc.5.bias', 'backbone.additional_blocks.1.0.bias', 'Constant_668', 'Constant_503', 'Constant_555', 'backbone.additional_blocks.1.2.weight', 'Unsqueeze_411', 'backbone.additional_blocks.1.2.bias', 'Constant_664', 'backbone.additional_blocks.2.2.weight', 'Constant_491', 'Constant_633', 'Constant_405', 'backbone.additional_blocks.2.2.bias', 'Constant_611', 'Constant_460', 'backbone.additional_blocks.3.0.weight', 'Constant_637', 'backbone.additional_blocks.3.0.bias', 'backbone.additional_blocks.3.2.weight', 'backbone.model.layer2.0.1.bn2.weight', 'Unsqueeze_368', 'backbone.additional_blocks.4.0.weight', 'backbone.additional_blocks.4.0.bias', 'backbone.model.layer1.4.0.bn1.running_mean', 'Constant_485', 'backbone.additional_blocks.4.2.weight', 'Constant_687', 'Constant_572', 'backbone.additional_blocks.4.2.bias', 'backbone.conf.0.weight', 'backbone.loc.0.bias', 'Unsqueeze_466', 'Constant_652', 'Constant_339', 'Constant_674', 'backbone.conf.0.bias', 'Constant_516', 'Constant_350', 'Constant_521', 'backbone.conf.4.weight', 'backbone.model.layer1.5.2.bn2.running_mean', 'backbone.model.layer2.0.1.bn2.bias', 'Unsqueeze_357', 'Constant_557', 'backbone.loc.1.weight', 'Unsqueeze_367', 'backbone.conf.1.weight', 'Constant_586', 'backbone.loc.2.weight', 'backbone.model.layer1.4.2.conv1.weight', 'backbone.conf.1.bias', 'Unsqueeze_378', 'Constant_517', 'Constant_383', 'Constant_537', 'Unsqueeze_389', 'Constant_497', 'Unsqueeze_390', 'Constant_592', 'backbone.conf.2.weight', 'Constant_542', 'Constant_394', 'backbone.model.layer2.0.0.conv1.weight', 'Constant_502', 'Constant_628', 'Unsqueeze_400', 'backbone.model.layer2.0.0.downsample.1.bias', 'Unsqueeze_401', 'backbone.loc.3.weight', 'Constant_416', 'backbone.model.layer2.0.3.bn1.running_mean', 'Unsqueeze_423', 'backbone.loc.4.weight', 'Constant_513', 'Constant_372', 'backbone.loc.4.bias', 'Constant_626', 'backbone.model.layer1.5.1.bn1.weight', 'Constant_427', 'Constant_489', 'Unsqueeze_433', 'Constant_595', 'Constant_547', 'backbone.conf.3.weight', 'Unsqueeze_434', 'Constant_479', 'backbone.conf.4.bias', 'Constant_449', 'Constant_606', 'Unsqueeze_455', 'Unsqueeze_456', 'backbone.model.layer2.0.5.bn2.weight', 'Constant_496', 'backbone.conf.5.bias', 'Unsqueeze_379', 'Constant_475', 'backbone.model.layer2.0.2.bn1.running_var', 'backbone.model.layer2.0.5.conv1.weight', 'Constant_477', 'backbone.model.layer2.0.4.bn1.bias', 'Constant_481', 'Constant_361', 'Constant_484', 'Constant_486', 'Constant_646', 'Constant_492', 'Constant_550', 'Constant_498', 'Constant_499', 'Constant_508']) are missing from input feed (['image']).\")

YOLOv3 (ONNX Hub, opset=12)

For split_init pass (Checker output):
C.check_model_path(\\nonnx.onnx_cpp2py_export.checker.ValidationError: Field \\'type\\' of \\'value_info\\' is required but missing.\\n'

For split_predict pass (Checker output):
C.check_model_path(\\nonnx.onnx_cpp2py_export.checker.ValidationError: Field \\'elem_type\\' of \\'type\\' is required but missing.\\n'

YOLOv3 (ONNX Hub, opset=10)

For split_init pass (Checker output):
C.check_model_path(\\nonnx.onnx_cpp2py_export.checker.ValidationError: Field \\'type\\' of \\'value_info\\' is required but missing.\\n'

For split_predict pass (Checker output):
C.check_model_path(\\nonnx.onnx_cpp2py_export.checker.ValidationError: Nodes in a graph must be topologically sorted, however input \\'TFNodes/yolo_evaluation_layer_1/arange_5/delta:0\\' of node: \\nname: add__252 OpType: Add\\n is not output of any previous nodes.\\n\\n==> Context: Bad node spec for node. Name: TFNodes/yolo_evaluation_layer_1/arange_1__246_loop OpType: Loop\\n'

This behavior occurs on classification models as well (e.g., DenseNet-121-7 and Inception-2).

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the split_init and split_predict failures with the cited SSD and YOLOv3 ONNX Hub models at opsets 10 and 12. Inspect the generated model parts and run the ONNX checker; done means both passes produce valid nets whose required inputs are satisfied.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
32/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.