scikit-learn / scikit-learn/scikit-learn
SGDClassifier.partial_fit mutates the model when sample_weight is all zeros
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 67.3k
- Forks
- 27.4k
- Avg merge
- 1d 15h
- Merged PRs (30d)
- 58
Description
Describe the bug and give evidence about its user-facing impact
SGDClassifier.partial_fit changes the model state even when the new batch has sample_weight=np.zeros(n_samples). In the code below, an all-zero-weight batch changes coef_, which affects subsequent predictions, and advances t_, which changes the learning-rate schedule for later partial_fit calls. I expect such a batch to behave like skipping it entirely. This matters to me because in weighted incremental-learning workflows, zero sample weights are a natural way to ignore samples or whole batches without changing the surrounding batching code. And the current behavior means that batches kept with zero weight are not equivalent to skipped batches, which silently changes the optimization path.
Related to #16298
Steps/Code to Reproduce
import numpy as np
from sklearn.datasets import make_classification
from sklearn.linear_model import SGDClassifier
def main():
X, y = make_classification(n_samples=40, random_state=1)
clf = SGDClassifier(random_state=0)
clf.partial_fit(X[:10], y[:10], classes=np.array([0, 1]))
coef_before = clf.coef_.copy()
t_before = clf.t_
clf.partial_fit(X[10:20], y[10:20], sample_weight=np.zeros(10))
coef_changed = bool(np.max(np.abs(clf.coef_ - coef_before)) > 0)
print("coef_changed:", coef_changed)
print("t_before:", t_before)
print("t_after:", clf.t_)
if __name__ == "__main__":
main()
Expected Results
coef_changed: False
t_before: 11.0
t_after: 11.0
Actual Results
coef_changed: True
t_before: 11.0
t_after: 21.0
Versions
System:
python: 3.10.12 (main, Jan 26 2026, 14:55:28) [GCC 11.4.0]
executable: /usr/bin/python3.10
machine: Linux-6.6.87.2-microsoft-standard-WSL2-x86_64-with-glibc2.35
Python dependencies:
sklearn: 1.7.2
pip: 26.0.1
setuptools: 59.6.0
numpy: 2.2.6
scipy: 1.15.3
Cython: None
pandas: None
matplotlib: None
joblib: 1.5.3
threadpoolctl: 3.6.0
Built with OpenMP: True
threadpoolctl info:
user_api: blas
internal_api: openblas
num_threads: 8
prefix: libscipy_openblas
filepath: /home/yang/.local/lib/python3.10/site-packages/numpy.libs/libscipy_openblas64_-56d6093b.so
version: 0.3.29
threading_layer: pthreads
architecture: Haswell
user_api: blas
internal_api: openblas
num_threads: 8
prefix: libscipy_openblas
filepath: /home/yang/.local/lib/python3.10/site-packages/scipy.libs/libscipy_openblas-68440149.so
version: 0.3.28
threading_layer: pthreads
architecture: Haswell
user_api: openmp
internal_api: openmp
num_threads: 8
prefix: libgomp
filepath: /home/yang/.local/lib/python3.10/site-packages/scikit_learn.libs/libgomp-a34b3233.so.1.0.0
version: None
Interest in fixing the bug
Yes, I would be interested in working on a PR if this is triaged as a bug. The issue seems to be that zero-weight samples are only zeroed out at the gradient-update stage, but are not skipped earlier in the optimization loop. The coefficient change disappears with penalty=None, which suggests the current behavior is tied to regularization being applied before zero-weight samples are fully skipped. My plan is to skip zero-weight samples earlier in the low-level SGD loop and add a test for an all-zero-weight partial_fit batch. Does that sound like the right direction?
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start at SGDClassifier.partial_fit and trace the low-level SGD loop where sample weights and regularization are applied. Reproduce the all-zero sample_weight case, then add a regression test confirming that coefficients and t_ remain unchanged, including the penalty-enabled path.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- numpy, python
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 58/100