dmlc / dmlc/xgboost

(scikit-learn api) (feature request) XGBRegressor with early stopping, dynamic eval_set

Open
#7,782 6 comments 0 reactions 0 assignees View on GitHub
feature-request
Dominant language
C++
Stars
28.8k
Forks
8.9k
Avg merge
1d 12h
Merged PRs (30d)
54

Description

XGBRegressor has the parameter eval_set, where you pass an evaluation set that regressor uses to perform early stopping.
Since this eval_set is fixed, when you do cross validation with n folds. In the n folds, eval_set is the same.

Would be cool to have the option that eval_set is dynamic at `fit` time, being a split from that particular fold. Find an example below.
IF you like the idea, happy to do a proper PR

```py
class XGBRegressorWithEarlyStop(XGBRegressor):
"""Wrapper of XGBRegressor with early stopping."""

def __init__(self, objective="reg:squarederror", early_stopping_rounds=5,
test_size=0.1, eval_metric='rmse', shuffle=False, **kwargs):
"""Init as super."""
self.early_stopping_rounds = early_stopping_rounds
self.test_size = test_size
self.eval_metric = eval_metric
self.shuffle = shuffle
super().__init__(objective=objective, **kwargs)

def fit(self, x, y, verbose=False, sample_weight=None):
"""Fit classifier."""
if sample_weight is not None:
x_train, x_val, y_train, y_val, w_train, w_val = train_test_split(
x, y, sample_weight,
test_size=self.test_size, shuffle=self.shuffle)
else:
x_train, x_val, y_train, y_val = train_test_split(
x, y,
test_size=self.test_size, shuffle=self.shuffle)
w_train, w_val = None, None
super().fit(x_train, y_train,
early_stopping_rounds=self.early_stopping_rounds,
eval_metric=self.eval_metric,
eval_set=[(x_val, y_val)],
verbose=verbose,
sample_weight=w_train,
sample_weight_eval_set=[w_val])
return self
```

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by tracing the XGBRegressor fit entry point and its scikit-learn cross-validation integration; the issue provides a wrapper example showing the requested behavior. Confirm how evaluation data is passed during each fold, then define done as supporting a fold-specific eval_set for early stopping without requiring the wrapper shown.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, scikit-learn
Domain
machine-learning
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.