Structure of multi-output trees
- Dominant language
- C++
- Stars
- 28.8k
- Forks
- 8.9k
- Avg merge
- 1d 12h
- Merged PRs (30d)
- 54
Description
In multi-output trees, leafs should have one value for each output, but if inspecting the trees, they appear to have more - guess multiplied by the depth or something like that. Example:
```python
import numpy as np
rng = np.random.default_rng(seed=123)
X = rng.standard_normal(size=(100,10))
y = rng.integers(5, size=X.shape[0])
import xgboost as xgb
model = xgb.train(
dtrain=xgb.DMatrix(X, y),
num_boost_round=1,
params={
"objective": "multi:softprob",
"max_depth": 2,
"multi_strategy": "multi_output_tree",
"num_class": 5,
},
)
import json
json.loads(
model.save_raw(raw_format="json")
)["learner"]["gradient_booster"]["model"]["trees"][-1]["leaf_weights"]
```
```
[0.13317914,
-0.12695906,
0.12011204,
-0.16318427,
0.10858103,
-0.0880465,
-0.083756156,
-0.13272737,
0.26806664,
0.04055968,
-0.13248305,
0.22173962,
-0.13717051,
-0.017899837,
-0.046972796,
0.01588887,
-0.03370758,
0.033632353,
0.0655839,
-0.057034157]
```
Has 20 values, yet the raw predictions output something with 5 values.
How do those numbers relate to the raw/margin predictions?
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with the provided Python reproducer, focusing on model.save_raw(raw_format="json") and the final tree's leaf_weights. Compare those values with the model's raw and margin predictions for the same inputs, tracing how multi-output tree leaves are interpreted. Done means explaining the 20-versus-5 value relationship or identifying the inconsistent serialization or prediction behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp, python
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 50/100