Retinanet failed to launch on MLPerf Inference v5.0

I wanted to test my video card and suddenly encountered an error while building tets. Previously (on older versions) this benchmark was running successfully.

The site I used to run the benchmark
https://docs.mlcommons.org/inference/benchmarks/object_detection/retinanet/

The commant

mlcr run-mlperf,inference,_find-performance,_full,_r5.0-dev \
   --model=retinanet \
   --implementation=nvidia \
   --framework=tensorrt \
   --category=edge \
   --scenario=Offline \
   --execution_mode=test \
   --device=cuda  \
   --docker --quiet \
   --test_query_count=500

GPU: 1 x H100
OS: Ubuntu 24.04.2

[2025-05-29 06:54:41,007 retinanet_graphsurgeon.py:264 INFO] Adding NMS layer nmsopt to the graph...
/home/mlcuser/.local/lib/python3.8/site-packages/torch/functional.py:504: UserWarning: torch.meshgrid: in an upcoming release, it will be required to pass the indexing argument. (Triggered internally at /home/cmuser/CM/repos/local/cache/ac4a8632ea8a437d/pytorch/aten/src/ATen/native/TensorShape.cpp:3516.)
  return _VF.meshgrid(tensors, **kwargs)  # type: ignore[attr-defined]
Traceback (most recent call last):
  File "/home/mlcuser/MLC/repos/local/cache/get-git-repo_mlperf-inferenc_3505ed3d/repo/closed/NVIDIA/code/actionhandler/base.py", line 78, in run
    success = self.handle()
  File "/home/mlcuser/MLC/repos/local/cache/get-git-repo_mlperf-inferenc_3505ed3d/repo/closed/NVIDIA/code/actionhandler/calibrate.py", line 62, in handle
    b.calibrate()
  File "/home/mlcuser/.local/lib/python3.8/site-packages/nvmitten/nvidia/builder.py", line 536, in calibrate
    self.mitten_builder.run(self.legacy_scratch, None)
  File "/home/mlcuser/MLC/repos/local/cache/get-git-repo_mlperf-inferenc_3505ed3d/repo/closed/NVIDIA/code/retinanet/tensorrt/Retinanet.py", line 379, in run
    network = self.create_network(self.builder, subnetwork_name=subnet_name)
  File "/home/mlcuser/MLC/repos/local/cache/get-git-repo_mlperf-inferenc_3505ed3d/repo/closed/NVIDIA/code/retinanet/tensorrt/Retinanet.py", line 223, in create_network
    success = parser.parse(onnx._serialize(model))
AttributeError: module 'onnx' has no attribute '_serialize'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/home/mlcuser/MLC/repos/local/cache/get-git-repo_mlperf-inferenc_3505ed3d/repo/closed/NVIDIA/code/main.py", line 231, in <module>
    main(main_args, DETECTED_SYSTEM)
  File "/home/mlcuser/MLC/repos/local/cache/get-git-repo_mlperf-inferenc_3505ed3d/repo/closed/NVIDIA/code/main.py", line 144, in main
    dispatch_action(main_args, config_dict, workload_setting)
  File "/home/mlcuser/MLC/repos/local/cache/get-git-repo_mlperf-inferenc_3505ed3d/repo/closed/NVIDIA/code/main.py", line 202, in dispatch_action
    handler.run()
  File "/home/mlcuser/MLC/repos/local/cache/get-git-repo_mlperf-inferenc_3505ed3d/repo/closed/NVIDIA/code/actionhandler/base.py", line 82, in run
    self.handle_failure()
  File "/home/mlcuser/MLC/repos/local/cache/get-git-repo_mlperf-inferenc_3505ed3d/repo/closed/NVIDIA/code/actionhandler/calibrate.py", line 68, in handle_failure
    raise RuntimeError("Calibration failed!")
RuntimeError: Calibration failed!
make: *** [Makefile:123: calibrate] Error 1
Traceback (most recent call last):
  File "/home/mlcuser/.local/bin/mlcr", line 8, in <module>
    sys.exit(mlcr())
  File "/home/mlcuser/.local/lib/python3.8/site-packages/mlc/main.py", line 86, in mlcr
    main()
  File "/home/mlcuser/.local/lib/python3.8/site-packages/mlc/main.py", line 273, in main
    res = method(run_args)
  File "/home/mlcuser/.local/lib/python3.8/site-packages/mlc/script_action.py", line 307, in run
    return self.call_script_module_function("run", run_args)
  File "/home/mlcuser/.local/lib/python3.8/site-packages/mlc/script_action.py", line 231, in call_script_module_function
    result = automation_instance.run(run_args)  # Pass args to the run method
  File "/home/mlcuser/MLC/repos/mlcommons@mlperf-automations/automation/script/module.py", line 226, in run
    r = self._run(i)
  File "/home/mlcuser/MLC/repos/mlcommons@mlperf-automations/automation/script/module.py", line 1857, in _run
    r = self._call_run_deps(prehook_deps, self.local_env_keys, local_env_keys_from_meta, env, state, const, const_state, add_deps_recursive,
  File "/home/mlcuser/MLC/repos/mlcommons@mlperf-automations/automation/script/module.py", line 3318, in _call_run_deps
    r = script._run_deps(deps, local_env_keys, env, state, const, const_state, add_deps_recursive, recursion_spaces,
  File "/home/mlcuser/MLC/repos/mlcommons@mlperf-automations/automation/script/module.py", line 3491, in _run_deps
    r = self.action_object.access(ii)
  File "/home/mlcuser/.local/lib/python3.8/site-packages/mlc/action.py", line 57, in access
    result = method(options)
  File "/home/mlcuser/.local/lib/python3.8/site-packages/mlc/script_action.py", line 307, in run
    return self.call_script_module_function("run", run_args)
  File "/home/mlcuser/.local/lib/python3.8/site-packages/mlc/script_action.py", line 231, in call_script_module_function
    result = automation_instance.run(run_args)  # Pass args to the run method
  File "/home/mlcuser/MLC/repos/mlcommons@mlperf-automations/automation/script/module.py", line 226, in run
    r = self._run(i)
  File "/home/mlcuser/MLC/repos/mlcommons@mlperf-automations/automation/script/module.py", line 1642, in _run
    r = self._call_run_deps(deps, self.local_env_keys, local_env_keys_from_meta, env, state, const, const_state, add_deps_recursive,
  File "/home/mlcuser/MLC/repos/mlcommons@mlperf-automations/automation/script/module.py", line 3318, in _call_run_deps
    r = script._run_deps(deps, local_env_keys, env, state, const, const_state, add_deps_recursive, recursion_spaces,
  File "/home/mlcuser/MLC/repos/mlcommons@mlperf-automations/automation/script/module.py", line 3491, in _run_deps
    r = self.action_object.access(ii)
  File "/home/mlcuser/.local/lib/python3.8/site-packages/mlc/action.py", line 57, in access
    result = method(options)
  File "/home/mlcuser/.local/lib/python3.8/site-packages/mlc/script_action.py", line 307, in run
    return self.call_script_module_function("run", run_args)
  File "/home/mlcuser/.local/lib/python3.8/site-packages/mlc/script_action.py", line 231, in call_script_module_function
    result = automation_instance.run(run_args)  # Pass args to the run method
  File "/home/mlcuser/MLC/repos/mlcommons@mlperf-automations/automation/script/module.py", line 226, in run
    r = self._run(i)
  File "/home/mlcuser/MLC/repos/mlcommons@mlperf-automations/automation/script/module.py", line 1642, in _run
    r = self._call_run_deps(deps, self.local_env_keys, local_env_keys_from_meta, env, state, const, const_state, add_deps_recursive,
  File "/home/mlcuser/MLC/repos/mlcommons@mlperf-automations/automation/script/module.py", line 3318, in _call_run_deps
    r = script._run_deps(deps, local_env_keys, env, state, const, const_state, add_deps_recursive, recursion_spaces,
  File "/home/mlcuser/MLC/repos/mlcommons@mlperf-automations/automation/script/module.py", line 3491, in _run_deps
    r = self.action_object.access(ii)
  File "/home/mlcuser/.local/lib/python3.8/site-packages/mlc/action.py", line 57, in access
    result = method(options)
  File "/home/mlcuser/.local/lib/python3.8/site-packages/mlc/script_action.py", line 307, in run
    return self.call_script_module_function("run", run_args)
  File "/home/mlcuser/.local/lib/python3.8/site-packages/mlc/script_action.py", line 243, in call_script_module_function
    raise ScriptExecutionError(f"Script {function_name} execution failed. Error : {error}")
mlc.script_action.ScriptExecutionError: Script run execution failed. Error : MLC script failed (name = app-mlperf-inference-nvidia, return code = 256)

Start with closed/NVIDIA/code/retinanet/tensorrt/Retinanet.py at line 223, where ONNX serialization is passed to the parser, and review the calibration path in closed/NVIDIA/code/actionhandler/calibrate.py. Reproduce the reported MLPerf Inference v5.0 command and confirm that calibration completes without the reported AttributeError and RuntimeError.