Monotone constraints in vector-leaf models
- Dominant language
- C++
- Stars
- 28.8k
- Forks
- 8.9k
- Avg merge
- 1d 12h
- Merged PRs (30d)
- 54
Description
Both monotone constraints and vector-leaf models can act as regularizers by limiting the allowed splits.
- Monotone constraints allow embedding domain knowledge into the model and prevent noise-derived splits.
- Vector Leaf models prefer splits that are good for many targets also preventing noise-derived splits.
However using both at the same time is sometimes impossible with the current API.
Consider the case where a feature has positive effect for target 1 and negative effect for target 2. This is a very common scenario, for example with multi-class classifiers. Because we can only supply a single monotone_constraints vector we can't describe these contradicting effects to the model. It is however trivial when training multiple trees (pass different monotone_constraints for each train).
Is there any planned work on making these two features work in tandem?
It would likely require receiving an vector of `monotone_constraints` vectors, one for each target (or equivalent dictionary format).
I don't know how easy would be to adapt the split selection algorithm to comply with those constraints.
Issue created from discussion [#12505](https://github.com/dmlc/xgboost/discussions/12505)
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by reviewing the existing monotone_constraints and vector-leaf model APIs, along with the split-selection algorithm referenced in the issue. No file or test is named; done would mean defining and implementing a supported way to apply different monotone constraints per target, with coverage for the conflicting-sign multi-target case.
Written by the indexing model from the issue text.
Assessment
- Domain
- machine-learning
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100