dmlc / dmlc/xgboost

XGBoosterReset initializes model state from an arbitrary cache DMatrix

Open
#12,457 3 comments 0 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
28.8k
Forks
8.9k
Avg merge
1d 12h
Merged PRs (30d)
54

Description

`XGBoosterCreate()` accepts a list of cache matrices but does not identify which is training data. If no training iteration occurs, `XGBoosterReset()` initializes model state from the first cached matrix.

This means cache ordering can change the model intercept and predictions.

### Reproduction

```python
import json
import numpy as np
import xgboost as xgb

X = np.zeros((4, 2), dtype=np.float32)
dtrain = xgb.DMatrix(X, label=np.full(4, 10.0))
deval = xgb.DMatrix(X, label=np.full(4, -5.0))
params = {"objective": "reg:squarederror", "nthread": 1}

def zero_tree(cache):
booster = xgb.Booster(params=params, cache=cache)
booster.reset()

config = json.loads(booster.save_config())
base_score = config["learner"]["learner_model_param"]["base_score"]
return base_score, booster.predict(dtrain)

print(zero_tree([dtrain, deval]))
print(zero_tree([deval, dtrain]))
```

Observed on current master:

```text
([1E1], [10, 10, 10, 10])
([-5E0], [-5, -5, -5, -5])
```

The zero-tree model changes solely because the cache order changes. R can trigger this because `xgb.train()` currently passes evaluation matrices before `dtrain`.

### Root cause

`Reset()` uses the first live constructor-cache matrix to initialize an unfitted learner and estimate its intercept. This contradicts its documented role of releasing runtime caches.

### Proposed direction

`Reset()` should preserve initialization state and never select training data.

The C API needs construction or initialization that explicitly receives:

- the training DMatrix;
- the complete parameter batch;
- custom-objective initialization policy.

Because the existing cache-based constructor cannot distinguish training from evaluation data, this bug may justify replacing it in the next C API compatibility window rather than maintaining two ambiguous construction paths.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with the XGBoosterCreate and XGBoosterReset behavior described in the issue, then run the provided Python reproduction with both cache orders and inspect save_config() and predict(). Done means an unfitted model keeps the same initialization state and predictions regardless of cache ordering, while reset still releases runtime caches.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp, python, r
Domain
api, backend-api-design, machine-learning
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.