microsoft / microsoft/onnxruntime

TreeEnsembleRegressor inference crashes when attribute data is stored externally

Open
#17,088 2 comments 0 reactions 1 assignee Claimed by @xadupre View on GitHub
core runtime
Dominant language
C++
Stars
21.9k
Forks
4.2k
Avg merge
4d 11h
Merged PRs (30d)
184

Description

### Describe the issue

Creating an inference session using a model with a `TreeEnsembleRegressor` operator that has attribute data stored externally crashes with an error like:

```
onnxruntime.capi.onnxruntime_pybind11_state.RuntimeException: [ONNXRuntimeError] : 6 : RUNTIME_EXCEPTION : Exception during initialization: [ShapeInferenceError] Cannot parse data from external tensors. Please load external data into raw data for tensor: target_weights
```

I've debugged this in the C++ library using a source build at commit `f17efb5c7b`, and the "ShapeInferenceError" is a little misleading as this doesn't seem related to shape inference, but is caused by `TreeEnsembleCommon::Init` calling `GetVectorAttrsOrDefault` for tensor attributes, which in turn calls `ONNX_NAMESPACE::ParseData(&proto)`, which tries to load data from the protobuf file and fails because the data is stored externally.

I can see there is a helper method for loading external data, `GetExtDataFromTensorProto` in `tensorprotoutils.h`, but it's not clear that there's an easy way to fix this as `TreeEnsembleCommon::Init` only has a reference to an `OpKernelInfo` instance and `GetExtDataFromTensorProto` needs an `Env` and `model_path`.

I'm not that familiar with the onnxruntime codebase though, maybe instead of fixing this within the operator implementation, the attribute data should be loaded as part of the general session initialization and the in-memory protobuf for the attributes modified so that the operator doesn't need to know the data was loaded externally.

### To reproduce

```python
import numpy as np
import onnx
from onnx import checker, helper, numpy_helper
from onnx.onnx_pb import TensorProto
import onnxruntime

# Create model

X = helper.make_tensor_value_info("X", TensorProto.FLOAT, ["N", 1])
Y = helper.make_tensor_value_info("Y", TensorProto.FLOAT, ["N", 1])

nodes_treeids = [0, 0, 0]
nodes_nodeids = [0, 1, 2]
nodes_modes = ["BRANCH_LT", "LEAF", "LEAF"]
nodes_truenodeids = [1, -1, -1]
nodes_falsenodeids = [2, -1, -1]
nodes_featureids = [0, -1, -1]
nodes_values = [0.5, np.NaN, np.NaN]
n_targets = 1
target_treeids = [0, 0]
target_nodeids = [1, 2]
target_ids = [0, 0]
target_weights = np.array([1.0, 2.0], dtype=np.float32)
target_weights_tensor = numpy_helper.from_array(target_weights, 'target_weights')

checker_context = checker.C.CheckerContext()
checker_context.ir_version = onnx.IR_VERSION
checker_context.opset_imports = {
"": onnx.defs.onnx_opset_version(),
"ai.onnx.ml": 3,
}

node = helper.make_node(
"TreeEnsembleRegressor",
domain="ai.onnx.ml",
inputs=["X"],
outputs=["Y"],
nodes_treeids=nodes_treeids,
nodes_nodeids=nodes_nodeids,
nodes_modes=nodes_modes,
nodes_truenodeids=nodes_truenodeids,
nodes_falsenodeids=nodes_falsenodeids,
nodes_featureids=nodes_featureids,
nodes_values=nodes_values,
n_targets=n_targets,
target_treeids=target_treeids,
target_nodeids=target_nodeids,
target_ids=target_ids,
target_weights_as_tensor=target_weights_tensor,
)
checker.check_node(node, ctx=checker_context)

graph_def = helper.make_graph(
nodes=[node],
name="test-model",
inputs=[X],
outputs=[Y])
checker.check_graph(graph_def, ctx=checker_context)

opset_import = helper.make_opsetid("ai.onnx.ml", 3)
model_def = helper.make_model(
graph_def,
opset_imports=[opset_import],
producer_name="onnx-example")

checker.check_model(model_def)

# Save model, storing tensor attributes in external data

model_path = "tree_model.onnx"

onnx.save_model(
model_def, model_path,
save_as_external_data=True,
location='data',
size_threshold=0,
convert_attribute=True)

# Run inference

session = onnxruntime.InferenceSession(model_path)
input = np.array([
[0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8],
], dtype=np.float32).T
output = session.run(["Y"], {"X": input})
```

### Urgency

_No response_

### Platform

Linux

### OS Version

Fedora 38

### ONNX Runtime Installation

Released Package

### ONNX Runtime Version or Commit ID

1.15.1

### ONNX Runtime API

Python

### Architecture

X64

### Execution Provider

Default CPU

### Execution Provider Library Version

_No response_

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.