dmlc / dmlc/xgboost

A better method to enforce monotonic constraints in regression and classification trees

Open
#7,073 1 comment 1 reaction 0 assignees View on GitHub
feature-request
Dominant language
C++
Stars
28.8k
Forks
8.9k
Avg merge
1d 12h
Merged PRs (30d)
54

Description

Here is a paper called: **A better method to enforce monotonic constraints in regression and classification trees**

https://arxiv.org/abs/2011.00986

It seems to suggest that standard monotonic constraints to tree based models constrain the model too tightly and harshly impact predictive accuracy. They propose two new methods to improve on the current methods. Here is how much better it can perform on the [The Adult](https://archive.ics.uci.edu/ml/datasets/adult) dataset. Monotonic constraints were on: age, education and hours_per_week. You could see how these features could be legally required to be monotonic in certain regulated spaces:

![image](https://user-images.githubusercontent.com/11645712/124277057-86bd3680-db3c-11eb-88cd-585891b9c8da.png)

Here are the examples of the changes added to LightGBM:
https://github.com/microsoft/LightGBM/pull/3264
https://github.com/microsoft/LightGBM/pull/2939
https://github.com/microsoft/LightGBM/pull/2770

The Parameters for the model are as follows. I feel this should give a good quick high-level explanation of how/why the new functionality might work:
https://github.com/microsoft/LightGBM/blob/aab8fc18a259ca9ca95db5bac203b81cb2e1ad49/docs/Parameters.rst#monotone_constraints_method

Here is a nice theoretical example:
![image](https://user-images.githubusercontent.com/11645712/124276718-10203900-db3c-11eb-8c61-af6021d43785.png)

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.