huggingface / huggingface/setfit

predict and predict_proba give conflicting predictions

Open
#587 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
Jupyter Notebook
Stars
2.8k
Forks
267
Avg merge
36m
Merged PRs (30d)
5

Description

This is somewhat related to #462.

**The problem:**

If I call `model.predict()`, I get a label. If I call `model.predict_proba()` and try to get the final string via `model.labels` or `model.id2label`, I get a different label.

**Why this happens:**

I think it's because the sklearn model head (where the probabilities are extracted from) might reorganize the labels, leading to an inconsistency between what the model thinks the order is and how the probs are returned.

**Workaround**

Doing `model.model_head.classes_[int(np.argmax(probs))]`. However, I think that's not the intended way since classes_ is a private attribute?

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.