apple / apple/coremltools

Sparse dict prediction with converted xgboost model.

Open
#1,016 0 comments 0 reactions 0 assignees View on GitHub
feature request trees
Dominant language
Python
Stars
5.4k
Forks
850
Avg merge
4d 5h
Merged PRs (30d)
10

Description

## ❓Question

#### Main problem

Hello,

I have a question considering predict() method from MLModel. I'd lilke to call MLModedl's predict() method providing a 'sparse' dict as an input (e.g. model has 3 features and I'd like to provide a dict with only one feature as an input) and I am unable to do so. I have described my attempts below.

I have a native xgboost binary classifier

```python

from sklearn import datasets
from xgboost import Booster, XGBClassifier, XGBRegressor

iris = datasets.load_iris()

X_raw = iris.data[iris.target != 2, :3].astype(float) # we only take the first 3 features.
y_raw = iris.target[iris.target != 2].astype(int)

# take first 5 observations from class 0 and first 2 from class 1
# taking more than 2 from class 1 will cause python to crash after converting
# model providing dict as 'feature_names' para meters:
# Process finished with exit code 139 (interrupted by signal 11: SIGSEGV)
X_raw_0 = X_raw[y_raw == 0, :][:5]
X_raw_1 = X_raw[y_raw == 1, :][:2]
y_raw_0 = y_raw[y_raw == 0][:5]
y_raw_1 = y_raw[y_raw == 1][:2]

X = np.array(
[[float(in_el) for in_el in el] for el in X_raw_0] +
[[float(in_el) for in_el in el] for el in X_raw_1])

y = [el for el in y_raw_0] + [el for el in y_raw_1]

xc = XGBClassifier(missing=np.nan)
xc.fit(X=X, y=y)

```

and I am converting it to mlmodel with:

```python

import coremltools as ct

conv_xgb = \
ct.converters.xgboost.convert(
xc, mode='classifier', feature_names=['f0', 'f1', 'f2'],
class_labels=[0, 1])

```

Afterwards I am trying to make all features optional (in order to make sparse dict prediction possible) with:

```python

conv_xgb_spec = conv_xgb._spec

for el in conv_xgb_spec.description.input:
setattr(el.type, 'isOptional', True)

conv_xgb_opt = ct.models.MLModel(conv_xgb_spec)

```

And finally I am trying to call predict with a sparse dict (a dict with some features 'missing' e.g. sparse dict {'f1': 0.0} instead of {'f0': numpy.nan, 'f1': 0.0, 'f2': numpy.nan}):

```python

sparse_dict = {'f1': 0.0}
conv_xgb_opt.predict(sparse_dict)

```

Which raises missing feature error:

_NSLocalizedDescription = "Feature 'f0' not provided."_

#### Alternative approaches

I did some alternative attempts:
1 use of *Imputer* object to fill in missing features but with no luck
2 use of *DictVectorizer* - it fills missing features with 0.0. Is there any way to make it fill missing features with numpy.nan?
3 change of *feature_names* parameter in convert() call

#### Change of feature_names parameter in convert() call

After changing convert() call:

```python

conv_xgb = \
ct.converters.xgboost.convert(
xc, mode='classifier', feature_names={'input_dict': [0, 1, 2]},
class_labels=[0, 1])

```
I am able to predict with sparse dict, getting identical results with native xgboost predict():

```python

res = predict({'input_dict': {'f1': 1.0}})

```

However this is only valid if training input is small (e.g. take a look at the X from the first chunk). If model is trained on larger data predict() with sparse dict ends in:

_Process finished with exit code 139 (interrupted by signal 11: SIGSEGV)_

Is it possible to make predictions on a 'sparse' dict ? If not, is it possible to make DictVectorizer or Imputer put np.nans in the input 'sparse' dict?

## System Information
macOS 10.15
coremltools == 3.4+ (attempts were made on 3.4 and 4.0)
sklearn == 0.19.2
python version == 3.7.9

Contributor guide

Open the contributing guide

Research direction

Start with the MLModel.predict call and the ct.converters.xgboost.convert path shown in the issue, reproducing both the missing-feature error and the sparse-dict segmentation fault with the provided versions. Done means establishing whether sparse dict prediction is supported and either documenting the limitation or resolving the reported failure without changing native xgboost results.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.