dmlc / dmlc/xgboost

Sparse-matrix binary feature set gives strange split values

Open
#12,448 3 comments 0 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
28.8k
Forks
8.9k
Avg merge
1d 12h
Merged PRs (30d)
54

Description

In the R package, the following code

```
library(xgboost)
data(agaricus.train, package='xgboost')
bst <- xgboost(x = agaricus.train$data, y = factor(agaricus.train$label),
max_depth = 2, learning_rate = 1, nrounds = 1,
objective = "binary:logistic")
xgb.plot.tree(bst)
```
gives a tree with splits annotated as `odor=none<2.00001001`, `stalk-root=club<2.00001001`, `spore-print-color=green<2.00001001`.

The features are all 0/1 variables and the data are in a compressed sparse matrix format. The plot is correct if you ignore the `<2.00001001` and just treat the splint as yes/no for eg `odor=none`. It isn't just in `xgb.plot.tree()`; when I save the tree as JSON the same number appears.

If I convert the compressed sparse matrix to a dense 0/1 matrix the problem goes away and the splits are eg `odor=none<1`. This was a bit confusing for my students (and me); it would be helpful if the output display for the splits was the same for dense or compressed sparse 0/1 matrices, or if there was clear documentation about what `<2.00001001` meant.

[In a previous version I used a few years ago for teaching the same class there was a similar phenomenon, but with a small negative number instead of 2.00001001]

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with the reproducible R example in the issue and compare tree output for the compressed sparse and dense matrices, both through xgb.plot.tree() and saved JSON. Done means sparse 0/1 splits display consistently with dense splits, or the meaning of the numeric threshold is clearly documented.

Written by the indexing model from the issue text.

Assessment

Tech stack
r
Domain
machine-learning
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.