apple / apple/coremltools

Malloc error converting sklearn decision tree

Open
#214 3 comments 0 reactions 0 assignees View on GitHub
bug scikit-learn trees
Dominant language
Python
Stars
5.4k
Forks
850
Avg merge
4d 5h
Merged PRs (30d)
10

Description

I've trained a model using scikit-learn's DecisionTreeClassifier on a dataset with 1,600,000 rows, 15 features, max_depth=7. When I try to convert using coremltools for a model over about 2 mb, I get the following error:

`malloc: *** error for object 0x7fb096a0f738: incorrect checksum for freed object - object was probably modified after being freed. *** set a breakpoint in malloc_error_break to debug`

I might be doing this part wrong, but when I try to log and debug, I get:

`Segmentation fault: 11`

To recreate:

```python
import pandas as pd
import numpy as np
import sklearn
import coremltools
import random
import string
from sklearn.tree import DecisionTreeClassifier

X = np.random.choice([0, 1], size=(15*1000,), p=[1./3, 2./3])
X = np.split(X,1000)

y = []
for i in range(0,1000):
x = ''.join(random.choice(string.lowercase) for x in range(5))
y.append(x)

clf = DecisionTreeClassifier()
clf.fit(X,y)

d = {'arr': X, 'str': y}
df = pd.DataFrame(data=d)

coreml_model = coremltools.converters.sklearn.convert(clf, "arr",
"str") ## malloc error
coreml_model.save('mymodel.mlmodel')
```

Contributor guide

Open the contributing guide

Research direction

Start with the provided Python reproduction and the coremltools.converters.sklearn.convert entry point, comparing the reported large model with the smaller example. Reproduce the malloc error or segmentation fault and identify a conversion outcome that completes without either failure for the DecisionTreeClassifier.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, scikit-learn
Domain
machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.