scikit-learn / scikit-learn/scikit-learn

A possible alternative to alpha at fit-time for GaussianProcessRegressor

Open
#7,975 8 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

module:gaussian_process
Dominant language
Python
Stars
67.3k
Forks
27.4k
Avg merge
1d 15h
Merged PRs (30d)
58

Description

I love the new GaussianProcessRegressor class, thanks so much. One thing that I think would make it a little easier for me to use/explain to others is if there were an alternative to the alpha option in the constructor to include the "weight" of individual samples. Here is a minimal example of the current approach:

import numpy as np, sklearn.gaussian_process, sklearn.gaussian_process.kernels

X = [[0], [1]]
y = [.5, .5]
se = np.array([.1, .5])

kernel = sklearn.gaussian_process.kernels.Matern(length_scale=1.0, nu=1.5)
gp = sklearn.gaussian_process.GaussianProcessRegressor(kernel=kernel, alpha=se**2, optimizer=None) # set data variance with alpha parameter _here_
gp.fit(X, y)  # then set data values _here_

The second row of data has 5x more variation than the first, and the GPR handles this beautifully (see notebook), but I find it aesthetically unappealing to set alpha in the constructor when the data that needs to match it is not set until the fit method is called.

I would prefer an approach where the alpha value was set in the fit function, such as

gp = sklearn.gaussian_process.GaussianProcessRegressor(kernel=kernel, optimizer=None)
gp.fit(X, y, alpha=se**2)

or, since it seems like sample_weight is used for doing this in LinearRegression and other places, it might be more consistent to use that instead of alpha:

gp.fit(X, y, sample_weight=se**-2)

I can potentially put together a pull request if this is a change that you are interested in. Thanks again!

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with GaussianProcessRegressor's constructor and fit method, then compare how LinearRegression handles sample_weight. Review the linked notebook and the existing alpha behavior to determine whether fit-time alpha or sample_weight is the intended API; done means per-sample variation can be supplied at fitting without setting it in the constructor and the relevant behavior is tested.

Written by the indexing model from the issue text.

Assessment

Tech stack
numpy, python
Domain
backend-api-design, machine-learning
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.