microsoft / microsoft/onnxruntime
TreeEnsembleRegressor inference crashes when attribute data is stored externally
- Dominant language
- C++
- Stars
- 21.9k
- Forks
- 4.2k
- Avg merge
- 4d 11h
- Merged PRs (30d)
- 184
Description
### Describe the issue
Creating an inference session using a model with a `TreeEnsembleRegressor` operator that has attribute data stored externally crashes with an error like:
```
onnxruntime.capi.onnxruntime_pybind11_state.RuntimeException: [ONNXRuntimeError] : 6 : RUNTIME_EXCEPTION : Exception during initialization: [ShapeInferenceError] Cannot parse data from external tensors. Please load external data into raw data for tensor: target_weights
```
I've debugged this in the C++ library using a source build at commit `f17efb5c7b`, and the "ShapeInferenceError" is a little misleading as this doesn't seem related to shape inference, but is caused by `TreeEnsembleCommon::Init` calling `GetVectorAttrsOrDefault` for tensor attributes, which in turn calls `ONNX_NAMESPACE::ParseData(&proto)`, which tries to load data from the protobuf file and fails because the data is stored externally.
I can see there is a helper method for loading external data, `GetExtDataFromTensorProto` in `tensorprotoutils.h`, but it's not clear that there's an easy way to fix this as `TreeEnsembleCommon::Init` only has a reference to an `OpKernelInfo` instance and `GetExtDataFromTensorProto` needs an `Env` and `model_path`.
I'm not that familiar with the onnxruntime codebase though, maybe instead of fixing this within the operator implementation, the attribute data should be loaded as part of the general session initialization and the in-memory protobuf for the attributes modified so that the operator doesn't need to know the data was loaded externally.
### To reproduce
```python
import numpy as np
import onnx
from onnx import checker, helper, numpy_helper
from onnx.onnx_pb import TensorProto
import onnxruntime
# Create model
X = helper.make_tensor_value_info("X", TensorProto.FLOAT, ["N", 1])
Y = helper.make_tensor_value_info("Y", TensorProto.FLOAT, ["N", 1])
nodes_treeids = [0, 0, 0]
nodes_nodeids = [0, 1, 2]
nodes_modes = ["BRANCH_LT", "LEAF", "LEAF"]
nodes_truenodeids = [1, -1, -1]
nodes_falsenodeids = [2, -1, -1]
nodes_featureids = [0, -1, -1]
nodes_values = [0.5, np.NaN, np.NaN]
n_targets = 1
target_treeids = [0, 0]
target_nodeids = [1, 2]
target_ids = [0, 0]
target_weights = np.array([1.0, 2.0], dtype=np.float32)
target_weights_tensor = numpy_helper.from_array(target_weights, 'target_weights')
checker_context = checker.C.CheckerContext()
checker_context.ir_version = onnx.IR_VERSION
checker_context.opset_imports = {
"": onnx.defs.onnx_opset_version(),
"ai.onnx.ml": 3,
}
node = helper.make_node(
"TreeEnsembleRegressor",
domain="ai.onnx.ml",
inputs=["X"],
outputs=["Y"],
nodes_treeids=nodes_treeids,
nodes_nodeids=nodes_nodeids,
nodes_modes=nodes_modes,
nodes_truenodeids=nodes_truenodeids,
nodes_falsenodeids=nodes_falsenodeids,
nodes_featureids=nodes_featureids,
nodes_values=nodes_values,
n_targets=n_targets,
target_treeids=target_treeids,
target_nodeids=target_nodeids,
target_ids=target_ids,
target_weights_as_tensor=target_weights_tensor,
)
checker.check_node(node, ctx=checker_context)
graph_def = helper.make_graph(
nodes=[node],
name="test-model",
inputs=[X],
outputs=[Y])
checker.check_graph(graph_def, ctx=checker_context)
opset_import = helper.make_opsetid("ai.onnx.ml", 3)
model_def = helper.make_model(
graph_def,
opset_imports=[opset_import],
producer_name="onnx-example")
checker.check_model(model_def)
# Save model, storing tensor attributes in external data
model_path = "tree_model.onnx"
onnx.save_model(
model_def, model_path,
save_as_external_data=True,
location='data',
size_threshold=0,
convert_attribute=True)
# Run inference
session = onnxruntime.InferenceSession(model_path)
input = np.array([
[0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8],
], dtype=np.float32).T
output = session.run(["Y"], {"X": input})
```
### Urgency
_No response_
### Platform
Linux
### OS Version
Fedora 38
### ONNX Runtime Installation
Released Package
### ONNX Runtime Version or Commit ID
1.15.1
### ONNX Runtime API
Python
### Architecture
X64
### Execution Provider
Default CPU
### Execution Provider Library Version
_No response_
Contributor guide
Assessment
This issue has not been assessed yet.