scikit-learn / scikit-learn/scikit-learn
Inconsistent behavior with bagging & base estimator class_weight
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 67.3k
- Forks
- 27.4k
- Avg merge
- 1d 15h
- Merged PRs (30d)
- 58
Description
Describe the bug
Bagging Classifier transforms y to indices between 0..N but does not impact the possible base_estimator.class_weight attribute making the fit method fails when using class_weight: the class could not be found.
Steps/Code to Reproduce
This is working fine:
from sklearn.datasets import make_classification
from sklearn.linear_model import PassiveAggressiveClassifier
X, y = make_classification(n_samples=10000, n_features=5, n_redundant=0, n_clusters_per_class=1, weights=[0.5])
# let's say y is not 0 or 1 but something else like 1 and 2 for instance:
y += 1
# now i fit a classifier with class_weight, works fine
PassiveAggressiveClassifier(class_weight={1: 1, 2: 1}).fit(X, y)
Now i'm just wrapping it around BaggingClassifier:
from sklearn.ensemble import BaggingClassifier
BaggingClassifier(PassiveAggressiveClassifier(class_weight={1: 1, 2: 1})).fit(X, y)
I'm getting:
ValueError: Class label 2 not present.
It seems it comes from here : https://github.com/scikit-learn/scikit-learn/blob/95119c13af77c76e150b753485c662b7c52a41a2/sklearn/ensemble/_bagging.py#L654
Because bagging transform y but then class_weight attribute of the base estimator is not aligned with the new labels.
Is it expected or this is a bug ?
Versions
System:
python: 3.7.9 (default, Feb 13 2021, 00:48:23) [GCC 10.2.1 20210110]
executable: /home/.../.pyenv/versions/3.7.9/bin/python3.7
machine: Linux-5.10.0-3-amd64-x86_64-with-debian-bullseye-sid
Python dependencies:
pip: 21.0.1
setuptools: 53.0.0
sklearn: 0.24.1
numpy: 1.20.1
scipy: 1.6.1
Cython: None
pandas: None
matplotlib: None
joblib: 1.0.1
threadpoolctl: 2.1.0
Built with OpenMP: True
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in sklearn/ensemble/_bagging.py around the referenced line and reproduce the failure with BaggingClassifier wrapping PassiveAggressiveClassifier and nonzero class labels. Trace how y is transformed and how the base estimator's class_weight is used; done means the reported example fits correctly without the class-label error, with regression coverage for the behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100