dmlc / dmlc/xgboost

Clarification on Trivial Splits

Open
#2,914 7 comments 0 reactions 1 assignee Claimed by @trivialfis View on GitHub
feature-request
Dominant language
C++
Stars
28.8k
Forks
8.9k
Avg merge
1d 12h
Merged PRs (30d)
54

Description

I have a question regarding how xgboost splits out missing values. The exact greedy and approximate tree methods tend to split out missing values by splitting at twice the maximum value on a feature. Whereas the histogram method splits at the minimum value of a feature. The factor of 2 seems consistent, so I don't think it has any dependence on the distribution of the feature.

Just for reference, I've been able to consistently see this kind of split in the latest release (v0.60).

I've found some threads discussing the treatment of missing data [2],[3],[4], but I'm interesting in understanding why 2?

So my question is the following:
Is the decision to exclusively split away missing values at twice the maximum of a variable a consequence/artifact/feature of the split proposal algorithm described in [1], or is it a feature of the implementation?

If it's the latter case, can this parameter be tuned?

[1] https://arxiv.org/abs/1603.02754
[2] https://github.com/dmlc/xgboost/issues/21
[3] https://github.com/dmlc/xgboost/issues/1112
[4] https://github.com/dmlc/xgboost/issues/1842

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.