dmlc / dmlc/xgboost

Usage of scalar-leaf trees with multi-output objectives

Open
#12,486 13 comments 0 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
28.8k
Forks
8.9k
Avg merge
1d 12h
Merged PRs (30d)
54

Description

In https://github.com/dmlc/xgboost/issues/12278 , it was mentioned:
> In one tree-per-class mode, one idea would be to recompute the gradients after each tree, instead of after each group of trees. This turns it into a coordinate descent algorithm which is much better at dealing with curvature.

Following this example:
https://xgboost.readthedocs.io/en/stable/tutorials/advanced_custom_obj.html

If setting parameters `num_boost_round=1` and `base_score=0` without any base_score, then calling something like:
```python
booster.inplace_predict(X, predict_type="margin")
```

I would expect from that comment and from the documentation to get a single tree making predictions for a single column of the output, but `.predict()` produces an output where each column has a non-zero value, even though the custom objective is called only once (verified by putting prints on it) and there is a single tree, as verified by:

```python
import json
trees = json.loads(booster.save_raw(raw_format="json"))["learner"]["gradient_booster"]["model"]["trees"]
len(trees)
```

What's happening there?

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with the advanced custom objective documentation and the discussion in issue 12278. Reproduce the case with num_boost_round=1, base_score=0, inplace_predict(..., predict_type="margin"), and the JSON tree-count check, then trace why predict() returns non-zero values for every output column. Done means explaining whether this behavior is expected or identifying the change needed.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp, python
Domain
machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.