Feature request: Send null features down both branches
- Dominant language
- C++
- Stars
- 28.8k
- Forks
- 8.9k
- Avg merge
- 1d 12h
- Merged PRs (30d)
- 54
Description
My understanding is that XGBoost currently handles null values as separate datatypes, and will split on whether the data is null or not before moving on to what the value is if not null. This is very reasonable if the existence of a null value has predictive power over the target variable or even if it is uncorrelated. But consider a demand forecasting problem in which a feature went missing around peak season. An XGBoost model built on this data could learn that missing values of this feature indicate high demand. If the feature being null were a boolean variable, we would of course discard it to prevent the model from considering this conclusion, but XGBoost's approach effectively forces the use of this feature. Since XGBoost is built on decision trees, it is in a unique position to handle null values with a [modified tree learning algorithm](http://mlwiki.org/index.php/Decision_Tree_(Data_Mining)#Handling_Missing_Values). The idea is null features should be treated as weighted leaves in both splits and weighed according to the proportion of leaves with non-null data in each branch.
Of course, this only works if non-null values of the feature generally provide intended predictive power. Going back to the forecasting example, if a feature went live during peak season, then went back offline, null values of the feature could still become an indicator of high demand. That said, dropping the feature entirely in this scenario is probably more sensible. Overall, I'd rather have the option of dropping a mostly useless feature than let a rare instance of its absence potentially spoil its predictive power should it go missing again.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.