scikit-learn / scikit-learn/scikit-learn

Inconsistent behavior with bagging & base estimator class_weight

Open
#19,665 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Bug module:ensemble
Dominant language
Python
Stars
67.3k
Forks
27.4k
Avg merge
1d 15h
Merged PRs (30d)
58

Description

Describe the bug

Bagging Classifier transforms y to indices between 0..N but does not impact the possible base_estimator.class_weight attribute making the fit method fails when using class_weight: the class could not be found.

Steps/Code to Reproduce

This is working fine:

from sklearn.datasets import make_classification
from sklearn.linear_model import PassiveAggressiveClassifier

X, y = make_classification(n_samples=10000, n_features=5, n_redundant=0, n_clusters_per_class=1, weights=[0.5])
# let's say y is not 0 or 1 but something else like 1 and 2 for instance:
y += 1
# now i fit a classifier with class_weight, works fine
PassiveAggressiveClassifier(class_weight={1: 1, 2: 1}).fit(X, y)

Now i'm just wrapping it around BaggingClassifier:

from sklearn.ensemble import BaggingClassifier

BaggingClassifier(PassiveAggressiveClassifier(class_weight={1: 1, 2: 1})).fit(X, y)

I'm getting:

ValueError: Class label 2 not present.

It seems it comes from here : https://github.com/scikit-learn/scikit-learn/blob/95119c13af77c76e150b753485c662b7c52a41a2/sklearn/ensemble/_bagging.py#L654

Because bagging transform y but then class_weight attribute of the base estimator is not aligned with the new labels.

Is it expected or this is a bug ?

Versions
 System:
    python: 3.7.9 (default, Feb 13 2021, 00:48:23)  [GCC 10.2.1 20210110]
executable: /home/.../.pyenv/versions/3.7.9/bin/python3.7
   machine: Linux-5.10.0-3-amd64-x86_64-with-debian-bullseye-sid

Python dependencies:
          pip: 21.0.1
   setuptools: 53.0.0
      sklearn: 0.24.1
        numpy: 1.20.1
        scipy: 1.6.1
       Cython: None
       pandas: None
   matplotlib: None
       joblib: 1.0.1
threadpoolctl: 2.1.0

Built with OpenMP: True

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in sklearn/ensemble/_bagging.py around the referenced line and reproduce the failure with BaggingClassifier wrapping PassiveAggressiveClassifier and nonzero class labels. Trace how y is transformed and how the base estimator's class_weight is used; done means the reported example fits correctly without the class-label error, with regression coverage for the behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.