GradientBoostedPropensityModel early-stopping split ignores random_state and is not stratified

GradientBoostedPropensityModel.fit() draws its early-stopping validation set with no random_state and no stratify (propensity.py:181-184):

if self.early_stop:
    X_train, X_val, y_train, y_val = train_test_split(
        X, y, test_size=stop_val_size
    )

a. random_state never reaches the split

_model (propensity.py:154-163) resolves the seed for the XGBClassifier as self.model_kwargs.get("random_state", 42), so GradientBoostedPropensityModel(random_state=42, early_stop=True) looks fully seeded. The validation set it early-stops against is drawn from the global numpy stream instead, so every fit() sees a different split, stops at a different iteration, and returns a different model.

import numpy as np
from causalml.propensity import GradientBoostedPropensityModel

rng = np.random.RandomState(42)
X = rng.normal(size=(1000, 10))
w = (0.8 * X[:, 0] + 0.5 * X[:, 1] + rng.normal(size=1000) > 0).astype(int)

for label, early_stop in [("early_stop=True ", True), ("early_stop=False", False)]:
    runs = []
    for _ in range(3):
        pm = GradientBoostedPropensityModel(random_state=42, early_stop=early_stop)
        runs.append(pm.fit_predict(X, w))
    same = all(np.array_equal(runs[0], r) for r in runs[1:])
    spread = max(np.abs(runs[0] - r).max() for r in runs[1:])
    print(f"{label} identical across 3 fits: {same}   max|diff| = {spread:.4f}")

early_stop=True  identical across 3 fits: False   max|diff| = 0.5466
early_stop=False identical across 3 fits: True   max|diff| = 0.0000

The early_stop=False line is the control: same seed, same data, bit-identical. That isolates the split as the only source of nondeterminism. A propensity score that moves by up to 0.55 between two identical calls propagates into every IPW/DR estimate built on it.

This is the same class of defect as #1029 (R-learner bootstraps). Notably the R-learner's own early-stopping split already threads the seed — rlearner.py:822-831 passes random_state=self.random_state to train_test_split. GradientBoostedPropensityModel is the only early-stopping split in the library that does not.

b. The split is not stratified on treatment

y here is the binary treatment indicator. Every other treatment split in the library stratifies on it:

• compute_r_residuals (propensity.py:317-320), whose docstring says "stratified on treatment so every fold retains both arms"
• LogisticRegressionPropensityModel's StratifiedKFold (propensity.py:113-121)

The early-stopping split is the exception. For imbalanced treatment the validation set's treated count swings widely, and at small n it can hold no treated units at all, which makes the early-stopping metric meaningless. Over 500 random splits at test_size=0.2:

This is adjacent to #1027, which tuned the other propensity models for imbalanced treatment.

Suggested fix

Both are the same call:

X_train, X_val, y_train, y_val = train_test_split(
    X,
    y,
    test_size=stop_val_size,
    random_state=self.model_kwargs.get("random_state", 42),
    stratify=y,
)

Reusing the self.model_kwargs.get("random_state", 42) expression from _model keeps the split and the classifier on the same seed, and leaves random_state=None genuinely random per the scikit-learn convention.

Observed on master (9a23278).

🤖 Generated with Claude Code

Start in propensity.py:154-184, reading how _model resolves random_state and how GradientBoostedPropensityModel.fit() creates the early-stopping split. Check the existing treatment-splitting patterns in propensity.py, including compute_r_residuals and LogisticRegressionPropensityModel. Done means seeded fits are repeatable and the validation split is stratified on treatment, while random_state=None remains nondeterministic.