rasbt / rasbt/mlxtend

How to use precomputed metric/kernels?

Open
#484 5 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
5.2k
Forks
916
Avg merge
17h 31m
Merged PRs (30d)
4

Description

What is the correct way of using precomputed metric (or kernel) with StackingCVClassifier and StackingCVRegressor functions? Below is a minimal modified version of your first StackingCVClassifier example code using precomputed metric with KNeighborsClassifier (three flavors). It won't work and gives an error:
"ValueError: Precomputed metric requires shape (n_queries, n_indexed). Got (51, 150) for 48 indexed."

from sklearn import datasets
from sklearn import model_selection
from sklearn.linear_model import LogisticRegression
from sklearn.neighbors import KNeighborsClassifier
from mlxtend.classifier import StackingCVClassifier
from sklearn.model_selection import GridSearchCV
import numpy as np

iris = datasets.load_iris()
X, y = iris.data[:, 1:3], iris.target

RANDOM_SEED = 42

clf1 = KNeighborsClassifier(n_neighbors=1,metric='precomputed')
clf2 = KNeighborsClassifier(n_neighbors=3,metric='precomputed')
clf3 = KNeighborsClassifier(n_neighbors=5,metric='precomputed')
lr = LogisticRegression()

np.random.seed(RANDOM_SEED)
sclf = StackingCVClassifier(classifiers=[clf1, clf2, clf3],
                            meta_classifier=lr)

print('3-fold cross validation:\n')

# create pre-computed distance matrix XX
n=X.shape[0]
XX=np.zeros((n,n))
for i in range(n):
    for j in range(n):
        XX[i,j] = np.mean(np.square((X[i,:]-X[j,:])))

# verify that GridSearchCV works
GS_clf = GridSearchCV(clf1, {'n_neighbors':[1,3,5,7,9,11]},cv=10)
GS_clf.fit(XX,y)
print(GS_clf.best_estimator_)

for clf, label in zip([clf1, clf2, clf3, sclf],
                      ['KNN1',
                       'KNN2',
                       'KNN3',
                       'StackingClassifier']):

    scores = model_selection.cross_val_score(clf, XX, y,cv=3, scoring='accuracy')
    print("Accuracy: %0.2f (+/- %0.2f) [%s]" % (scores.mean(), scores.std(), label))

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the failure at the StackingCVClassifier entry point with the provided Iris example and precomputed distance matrix, then compare it with StackingCVRegressor. Done should be established by determining whether precomputed metrics can be supported without the (51, 150) versus 48 indexed shape error, and by covering the behavior with tests.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, scikit-learn
Domain
machine-learning
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.