Sparse-matrix binary feature set gives strange split values
- Dominant language
- C++
- Stars
- 28.8k
- Forks
- 8.9k
- Avg merge
- 1d 12h
- Merged PRs (30d)
- 54
Description
In the R package, the following code
```
library(xgboost)
data(agaricus.train, package='xgboost')
bst <- xgboost(x = agaricus.train$data, y = factor(agaricus.train$label),
max_depth = 2, learning_rate = 1, nrounds = 1,
objective = "binary:logistic")
xgb.plot.tree(bst)
```
gives a tree with splits annotated as `odor=none<2.00001001`, `stalk-root=club<2.00001001`, `spore-print-color=green<2.00001001`.
The features are all 0/1 variables and the data are in a compressed sparse matrix format. The plot is correct if you ignore the `<2.00001001` and just treat the splint as yes/no for eg `odor=none`. It isn't just in `xgb.plot.tree()`; when I save the tree as JSON the same number appears.
If I convert the compressed sparse matrix to a dense 0/1 matrix the problem goes away and the splits are eg `odor=none<1`. This was a bit confusing for my students (and me); it would be helpful if the output display for the splits was the same for dense or compressed sparse 0/1 matrices, or if there was clear documentation about what `<2.00001001` meant.
[In a previous version I used a few years ago for teaching the same class there was a similar phenomenon, but with a small negative number instead of 2.00001001]
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with the reproducible R example in the issue and compare tree output for the compressed sparse and dense matrices, both through xgb.plot.tree() and saved JSON. Done means sparse 0/1 splits display consistently with dense splits, or the meaning of the numeric threshold is clearly documented.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- r
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 55/100