scikit-learn / scikit-learn/scikit-learn

SGDClassifier.partial_fit mutates the model when sample_weight is all zeros

Open
#33,436 3 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Bug
Dominant language
Python
Stars
67.3k
Forks
27.4k
Avg merge
1d 15h
Merged PRs (30d)
58

Description

Describe the bug and give evidence about its user-facing impact

SGDClassifier.partial_fit changes the model state even when the new batch has sample_weight=np.zeros(n_samples). In the code below, an all-zero-weight batch changes coef_, which affects subsequent predictions, and advances t_, which changes the learning-rate schedule for later partial_fit calls. I expect such a batch to behave like skipping it entirely. This matters to me because in weighted incremental-learning workflows, zero sample weights are a natural way to ignore samples or whole batches without changing the surrounding batching code. And the current behavior means that batches kept with zero weight are not equivalent to skipped batches, which silently changes the optimization path.

Related to #16298

Steps/Code to Reproduce
import numpy as np
from sklearn.datasets import make_classification
from sklearn.linear_model import SGDClassifier


def main():
    X, y = make_classification(n_samples=40, random_state=1)

    clf = SGDClassifier(random_state=0)
    clf.partial_fit(X[:10], y[:10], classes=np.array([0, 1]))
    coef_before = clf.coef_.copy()
    t_before = clf.t_

    clf.partial_fit(X[10:20], y[10:20], sample_weight=np.zeros(10))

    coef_changed = bool(np.max(np.abs(clf.coef_ - coef_before)) > 0)
    print("coef_changed:", coef_changed)
    print("t_before:", t_before)
    print("t_after:", clf.t_)


if __name__ == "__main__":
    main()
Expected Results

coef_changed: False
t_before: 11.0
t_after: 11.0

Actual Results

coef_changed: True
t_before: 11.0
t_after: 21.0

Versions
System:
    python: 3.10.12 (main, Jan 26 2026, 14:55:28) [GCC 11.4.0]
executable: /usr/bin/python3.10
   machine: Linux-6.6.87.2-microsoft-standard-WSL2-x86_64-with-glibc2.35

Python dependencies:
      sklearn: 1.7.2
          pip: 26.0.1
   setuptools: 59.6.0
        numpy: 2.2.6
        scipy: 1.15.3
       Cython: None
       pandas: None
   matplotlib: None
       joblib: 1.5.3
threadpoolctl: 3.6.0

Built with OpenMP: True

threadpoolctl info:
       user_api: blas
   internal_api: openblas
    num_threads: 8
         prefix: libscipy_openblas
       filepath: /home/yang/.local/lib/python3.10/site-packages/numpy.libs/libscipy_openblas64_-56d6093b.so
        version: 0.3.29
threading_layer: pthreads
   architecture: Haswell

       user_api: blas
   internal_api: openblas
    num_threads: 8
         prefix: libscipy_openblas
       filepath: /home/yang/.local/lib/python3.10/site-packages/scipy.libs/libscipy_openblas-68440149.so
        version: 0.3.28
threading_layer: pthreads
   architecture: Haswell

       user_api: openmp
   internal_api: openmp
    num_threads: 8
         prefix: libgomp
       filepath: /home/yang/.local/lib/python3.10/site-packages/scikit_learn.libs/libgomp-a34b3233.so.1.0.0
        version: None
Interest in fixing the bug

Yes, I would be interested in working on a PR if this is triaged as a bug. The issue seems to be that zero-weight samples are only zeroed out at the gradient-update stage, but are not skipped earlier in the optimization loop. The coefficient change disappears with penalty=None, which suggests the current behavior is tied to regularization being applied before zero-weight samples are fully skipped. My plan is to skip zero-weight samples earlier in the low-level SGD loop and add a test for an all-zero-weight partial_fit batch. Does that sound like the right direction?

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start at SGDClassifier.partial_fit and trace the low-level SGD loop where sample weights and regularization are applied. Reproduce the all-zero sample_weight case, then add a regression test confirming that coefficients and t_ remain unchanged, including the penalty-enabled path.

Written by the indexing model from the issue text.

Assessment

Tech stack
numpy, python
Domain
machine-learning
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
58/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.