dmlc / dmlc/xgboost

Suggestion for new hyperparameter: Regularize tree depth

Open
#11,845 6 comments 0 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
28.8k
Forks
8.9k
Avg merge
1d 12h
Merged PRs (30d)
54

Description

Currently, xgboost uses a hard cap on the tree-depth set by the "max_depth" parameter and essentially treats every tree-depth level the same. It came to my mind that a softer cap + preferring lower tree-depths might be more appropriate in some cases.

### Motivation
The general intuition is that a-priori one could assume that first-order and lower-order feature interaction effects are generally higher in absolute terms and contain more information than higher-order feature interactions/tree depths. This would be in line with the usual testing and fitting strategy in the ANOVA literature, where one first adds the first order or main effects, then first-order interactions and so on.

### Proposed Change
Specifically, the proposal would be to shrink or decay the learning rate within an individual tree by tree depth. If the corresponding parameter (say "decay") is e.g. .8, one would shrink the learning rate (or shrink the delta step) by a factor of .8. If the learning rate is .1, it would be .08 for all splits at tree depth 2, .064 for all splits at tree depth 3 and so on.

One might argue that one could achieve a similar thing with adjusting some regularization parameter such as "lambda", but that doesn't seem to be the case; the suggestion delivers very promising results.

### Experimental Evidence
In particular, I quickly implemented the change using the rpart package in R (using less than 10 loc) and implemented a boosting algorithm. I don't have the computational means at hand to do extensive hyperparameter optimization with inner and outer cross-validation rounds, but I used a simpler scheme to assess the effectiveness of the proposal: I chose 10 regression datasets from TabArena (https://arxiv.org/pdf/2506.16791; https://github.com/autogluon/tabarena) that employ RMSE as evaluation criterion. For each of the 10 data sets 10 train-test splits were done (2/3 train, 1/3 test). For xgboost*, n_trees, max_depth and lambda were optimized on the following grid:
```r
hp_grid_trees <- c(20, 50, 75, 100, 125, 150, 200, 300, 400, 600, 800, 1000, 1250, 1500, 2000, 2500, 3000, 4000, 5000)
hp_tree_depth <- 4:12
hp_lambda <- c(0.01, 0.1, 0.5, 1, 2, 4, 8)
```
For the proposal** the following parameter grid was chosen:
```r
hp_grid_trees <- c(20, 50, 75, 100, 125, 150, 200, 300, 400, 600, 800, 1000, 1250, 1500, 2000, 2500, 3000, 4000, 5000)
hp_decay <- c(0.95, 0.9, 0.85, 0.8, 0.75, 0.7, 0.65, 0.6, 0.55, 0.5)
```

To prove the effectivness and for computational difference tuning was done differently: For xgboost, for each train/test split, the best solution on the grid was chosen. For the proposal, the optimization was done once on a 0th train/test split and the best parameter set was then used in all 10 train/test splits.
Clearly, this gives xgboost a significant advantage (and wouldn't be best practice as it leads to overfitting) together with an additional hyperparameter. However, the proposed method still beats classic xgboost in 9 out of 10 datasets (average RMSE):

|dataset | xgboost_best | gbt with new hyperparameter |
|-------------| ------------- | ------------- |
|wine_quality| 0.621 | 0.614 |
|superconductivity| 9.468 | 9.391 |
|miami_housing| 84656 | 82243 |
|houses| 0.2187 | 0.2127 |
|healthcare_insurance_expenses| 4489.8 | 4489.5 |
|diamonds| 566.66 | 528.2 |
|concrete_compressive_strength| 4.145 | 4.084 |
|airfoil_self_noise| 1.471 | 1.454 |
|QSAR_fish_toxicity| 0.855 | 0.878 |
|Another-Dataset-on-used-Fiat-500| 741.1 | 735.9 |

### Drawbacks:
Higher computational and memory needs as max_depth should be set to at least 12 (or better 14/15) and mostly more trees are needed (though this might be somewhat offset by a higher learning rate).

### Settings/Environments
* xgboost package version 1.7.11.1 in R; learning rate = 0.1, tree_method = "exact", objective = "reg:squarederror"
** rpart package version 4.1.23 in R; learning rate = 0.1, maxdepth = 15, minsplit and minbucket = 1, cp = 0
Without using the new hyperparameter, both approaches are comparable as they yield very similar evaluation results

Contributor guide

No contributing guide indexed for this repository

Research direction

No repository files or tests are named. Start by locating the existing max_depth and learning-rate handling, then compare the proposed depth-based decay with the R/rpart experiment and determine the required design. Done should include an agreed parameter behavior, implementation, tests, and benchmark evidence.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp, r
Domain
machine-learning
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.