dmlc / dmlc/xgboost

Make multiclass base_score consistently use response-space values

Open
#12,426 3 comments 0 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
28.8k
Forks
8.9k
Avg merge
1d 12h
Merged PRs (30d)
54

Description

Multiclass handles base_score differently from other objectives with an invertible link.

For logistic, Gamma, Poisson, and Tweedie objectives:

1. InitEstimation returns a response-space value.
2. ProbToMargin converts it to the internal margin.
3. A user-provided base_score is interpreted in response space.

Multiclass instead converts estimated class probabilities into centered logits directly inside InitEstimation and does not implement ProbToMargin. Consequently, a user-provided vector base_score is interpreted as logits rather than class probabilities.

Multiclass has a suitable canonical conversion despite softmax being invariant to a constant:

margin[i] = log(p[i]) - mean(log(p))

Proposed changes:

- Return class probabilities from multiclass InitEstimation.
- Implement ProbToMargin using clamped, centered log probabilities.
- Add tests comparing response-space base_score with the equivalent raw base_margin.
- Handle compatibility with existing models, which may store vector-valued multiclass base_score as already-transformed margins.

This would make multiclass consistent with the documented base_score convention while retaining a deterministic representative of the equivalent softmax margins.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by locating the multiclass InitEstimation implementation and the objective interface for ProbToMargin. Review existing base_score and base_margin tests, then trace how older vector-valued multiclass models are loaded. Done means response-space base_score matches the equivalent raw base_margin, with compatibility tests covering existing models.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
machine-learning
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.