scikit-learn / scikit-learn/scikit-learn

Bug in utils/multiclass.py/_ovr_decision_function

Open
#27,973 5 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Bug
Dominant language
Python
Stars
67.3k
Forks
27.4k
Avg merge
1d 15h
Merged PRs (30d)
58

Description

Describe the workflow you want to enable

Dear scikit learn developpers,

I think the implementation of _ovr_decision_function in utils
/multiclass.py doesn't work properly when the parameter confidences is probability. While as the documentation suggests, it can be a probability .

confidences : array-like of shape (n_samples, n_classifiers)
        Decision functions or predicted probabilities for positive class
        for each binary classifier.

The problem is the following two lines of codes

sum_of_confidences[:, i] -= confidences[:, k]
sum_of_confidences[:, j] += confidences[:, k]

In this context, there is a binary classifier for class i vs j. And j is the positive class.

If confidences is "decision_function", then it works. Because if "decision funtion" is negative, it means the classifier thinks the negatve class i is more possible. And the -= will increase the sum_of_confidences of i, and decrease the sum_of_confidences of j.

However, if confidences is "probability", it doesn't work. Because probability is always greater than zero. So the sum_of_confidences of i will always decrease, even when i is more likely to happend (prob of j < 0.5).

Describe your proposed solution

"decision_function" is centered at 0, while "probability" is centerd at 0.5. These two cases should be handled seperately.

Describe alternatives you've considered, if relevant

No response

Additional context

No response

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in utils/multiclass.py at _ovr_decision_function and inspect its documentation and callers to understand how decision functions and probabilities are passed. Check the existing multiclass utility tests, then define coverage for both input forms; done means the aggregation handles probability inputs consistently with the documented behavior without regressing decision-function inputs.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.