dmlc / dmlc/xgboost

Monotone constraints in vector-leaf models

Open
#12,506 0 comments 0 reactions 0 assignees View on GitHub
feature-request
Dominant language
C++
Stars
28.8k
Forks
8.9k
Avg merge
1d 12h
Merged PRs (30d)
54

Description

Both monotone constraints and vector-leaf models can act as regularizers by limiting the allowed splits.
- Monotone constraints allow embedding domain knowledge into the model and prevent noise-derived splits.
- Vector Leaf models prefer splits that are good for many targets also preventing noise-derived splits.

However using both at the same time is sometimes impossible with the current API.

Consider the case where a feature has positive effect for target 1 and negative effect for target 2. This is a very common scenario, for example with multi-class classifiers. Because we can only supply a single monotone_constraints vector we can't describe these contradicting effects to the model. It is however trivial when training multiple trees (pass different monotone_constraints for each train).

Is there any planned work on making these two features work in tandem?
It would likely require receiving an vector of `monotone_constraints` vectors, one for each target (or equivalent dictionary format).
I don't know how easy would be to adapt the split selection algorithm to comply with those constraints.

Issue created from discussion [#12505](https://github.com/dmlc/xgboost/discussions/12505)

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reviewing the existing monotone_constraints and vector-leaf model APIs, along with the split-selection algorithm referenced in the issue. No file or test is named; done would mean defining and implementing a supported way to apply different monotone constraints per target, with coverage for the conflicting-sign multi-target case.

Written by the indexing model from the issue text.

Assessment

Domain
machine-learning
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.