scikit-learn / scikit-learn/scikit-learn

Call to estimator.score() in RANSAC even if the produced input data set is too small - fix proposed

Open
#32,908 17 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Bug Needs Investigation
Dominant language
Python
Stars
67.3k
Forks
27.4k
Avg merge
1d 15h
Merged PRs (30d)
58

Description

Describe the bug

By sampling radomly from a distribution of x,y values, the RANSAC algorithm tries to find the line which describes best most of the data (while ignoring outliers). For deciding, how well a certain line describes the data, estimator.score() is called in line 526 of _ransac.py.

    525 # score of inlier data set
--> 526 score_subset = estimator.score(
    527     X_inlier_subset,
    528     y_inlier_subset,
    529     **score_params_inlier_subset,
    530 )

However, estimator.score() is called even if the inlier data set contains only a single data point, which then triggers a warning down the road:

...\Lib\site-packages\sklearn\metrics\_regression.py:1283, in r2_score(y_true, y_pred, sample_weight, multioutput, force_finite)
   1281 if _num_samples(y_pred) < 2:
   1282     msg = "R^2 score is not well-defined with less than two samples."
-> 1283     warnings.warn(msg, UndefinedMetricWarning)
   1284     return float("nan")

From my point of view, the code should not call this function if the input data set is not valid.

Unfortunately, I cannot provide a minimum example code as due to the random sampling the warning does only rarely occur. As far as I understand, the situation occurs if in previous iterations there was already a data set with more than 1 inlier data point drawn. Otherwise, line 526 is not reached due to:

510  # less inliers -> skip current random sample
511  if n_inliers_subset < n_inliers_best:
512     self.n_skips_no_inliers_ += 1
513     continue

If propose to modify this line to:

511 if ((n_inliers_subset < n_inliers_best) or (n_inliers_subset < 2)):

At least for me, this does fix the situation that generates the warnings.

Steps/Code to Reproduce
            # less inliers -> skip current random sample
            if n_inliers_subset < n_inliers_best: # proposed fix: if ((n_inliers_subset < n_inliers_best) or (n_inliers_subset < 2)):
                self.n_skips_no_inliers_ += 1
                continue

            # extract inlier data set
            inlier_idxs_subset = sample_idxs[inlier_mask_subset]
            X_inlier_subset = X[inlier_idxs_subset]
            y_inlier_subset = y[inlier_idxs_subset]

            # cut `fit_params` down to `inlier_idxs_subset`
            score_params_inlier_subset = _check_method_params(
                X, params=routed_params.estimator.score, indices=inlier_idxs_subset
            )

            # score of inlier data set
            score_subset = estimator.score(
                X_inlier_subset,
                y_inlier_subset,
                **score_params_inlier_subset,
            )
Expected Results

Expected result: RANSAC algorithm should not make call to functions with invalid data that cause warnings.

Actual Results
...\Lib\site-packages\sklearn\metrics\_regression.py:1283, in r2_score(y_true, y_pred, sample_weight, multioutput, force_finite)
   1281 if _num_samples(y_pred) < 2:
   1282     msg = "R^2 score is not well-defined with less than two samples."
-> 1283     warnings.warn(msg, UndefinedMetricWarning)
   1284     return float("nan")
Versions
import sklearn; sklearn.show_versions()

System:
    python: 3.13.5 | packaged by Anaconda, Inc. | (main, Jun 12 2025, 16:37:03) [MSC v.1929 64 bit (AMD64)]
executable: C:\ProgramData\anaconda3\envs\E134\python.exe
   machine: Windows-11-10.0.26200-SP0

Python dependencies:
      sklearn: 1.7.2
          pip: 25.2
   setuptools: 78.1.1
        numpy: 2.3.2
        scipy: 1.16.1
       Cython: None
       pandas: 2.3.2
   matplotlib: 3.10.6
       joblib: 1.5.2
threadpoolctl: 3.6.0

Built with OpenMP: True

threadpoolctl info:
       user_api: blas
   internal_api: openblas
    num_threads: 12
         prefix: libscipy_openblas
       filepath: C:\ProgramData\anaconda3\envs\E134\Lib\site-packages\numpy.libs\libscipy_openblas64_-860d95b1c38e637ce4509f5fa24fbf2a.dll
        version: 0.3.30
threading_layer: pthreads
   architecture: SkylakeX

       user_api: blas
   internal_api: openblas
    num_threads: 12
         prefix: libscipy_openblas
       filepath: C:\ProgramData\anaconda3\envs\E134\Lib\site-packages\scipy.libs\libscipy_openblas-6b2103f2ae4d8547998b5d188e9801fb.dll
        version: 0.3.28
threading_layer: pthreads
   architecture: SkylakeX

       user_api: openmp
   internal_api: openmp
    num_threads: 12
         prefix: vcomp
       filepath: C:\ProgramData\anaconda3\envs\E134\Lib\site-packages\sklearn\.libs\vcomp140.dll
        version: None

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Read _ransac.py around lines 510-530, focusing on the inlier-count check before estimator.score(). Reproduce or isolate a one-point inlier subset and add regression coverage that confirms invalid subsets do not trigger estimator scoring or UndefinedMetricWarning. Done means RANSAC avoids scoring datasets that are too small while preserving normal scoring behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.