Usage of scalar-leaf trees with multi-output objectives
- Dominant language
- C++
- Stars
- 28.8k
- Forks
- 8.9k
- Avg merge
- 1d 12h
- Merged PRs (30d)
- 54
Description
In https://github.com/dmlc/xgboost/issues/12278 , it was mentioned:
> In one tree-per-class mode, one idea would be to recompute the gradients after each tree, instead of after each group of trees. This turns it into a coordinate descent algorithm which is much better at dealing with curvature.
Following this example:
https://xgboost.readthedocs.io/en/stable/tutorials/advanced_custom_obj.html
If setting parameters `num_boost_round=1` and `base_score=0` without any base_score, then calling something like:
```python
booster.inplace_predict(X, predict_type="margin")
```
I would expect from that comment and from the documentation to get a single tree making predictions for a single column of the output, but `.predict()` produces an output where each column has a non-zero value, even though the custom objective is called only once (verified by putting prints on it) and there is a single tree, as verified by:
```python
import json
trees = json.loads(booster.save_raw(raw_format="json"))["learner"]["gradient_booster"]["model"]["trees"]
len(trees)
```
What's happening there?
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with the advanced custom objective documentation and the discussion in issue 12278. Reproduce the case with num_boost_round=1, base_score=0, inplace_predict(..., predict_type="margin"), and the JSON tree-count check, then trace why predict() returns non-zero values for every output column. Done means explaining whether this behavior is expected or identifying the change needed.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp, python
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100