Unexpected behavior with prediction on pools with baseline when loss_function has link function
- Dominant language
- C++
- Stars
- 9.1k
- Forks
- 1.3k
- PR merge metrics
- No merged PRs in 30d
Description
**Problem**: Unexpected results emerge when predicting on pools that contain a baseline, when the model objective has a link function (e.g., Poisson, Tweedie)
**catboost version**: 0.23.2
**Operating System**: Centos 7
**CPU**: Intel(R) Xeon(R) Gold 6130 CPU @ 2.10GHz
**GPU**: Tesla P100
```python
import numpy as np
import pandas as pd
from catboost import CatBoostRegressor, Pool
base = np.array([1.0, 2.0, 0.5, 0.0, 0.3, 0.0])
multiplier = np.array([0.2, 0.5, 2.0, 0.5, 2.0, 0.2])
train_label = base * multiplier
train_x = pd.DataFrame({'feature': ['a', 'b', 'c', 'b', 'c', 'a']})
def get_regressor():
return CatBoostRegressor(iterations = 3,
loss_function = "Poisson",
random_seed = 125,
depth=4)
# Learn baseline
tr_pool_1 = Pool(data=train_x, label=train_label, cat_features=['feature'])
model1 = get_regressor()
model1.fit(tr_pool_1, verbose=False)
# Learn with the above prediction as baseline
baseline = model1.predict(train_x, prediction_type="RawFormulaVal") # Not sure this prediction_type should be taken
tr_pool_2 = Pool(data=train_x, label=train_label, cat_features=['feature'], baseline=baseline)
model2 = get_regressor()
model2.fit(tr_pool_2, verbose=False)
print("Test cases")
eps = 1e-4
print("Case 1:")
print("Expected: predict on train_x, prediction_type=RawFormulaVal should be np.log(prediction_type=None)")
print("Result:", (np.abs(model2.predict(train_x) - np.exp(model2.predict(train_x, prediction_type="RawFormulaVal"))) <= eps).all())
# Result: True
print("Case 2:")
print("Expected: (RawFormulaVal prediction on tr_pool_2) should be np.log(Default prediction on tr_pool_2)")
print("Result:", (np.abs(model2.predict(tr_pool_2) - np.exp(model2.predict(tr_pool_2, prediction_type="RawFormulaVal"))) <= eps).all())
# Result: False
print("Case 3:")
print("Expected: (model 2 RawFormulaVal prediction on train_x) + (baseline) should be (model 2 RawFormulaVal prediction on tr_pool_2)")
print("Result:", (np.abs(model2.predict(train_x, prediction_type="RawFormulaVal") + baseline - model2.predict(tr_pool_2, prediction_type="RawFormulaVal")) <= eps).all())
# Result: True
print("Case 4:")
print("Expected: (model 2 default prediction on train_x) * exp(baseline) should be (model 2 default prediction on tr_pool_2)")
print("Result:", (np.abs(model2.predict(train_x) * np.exp(baseline) - model2.predict(tr_pool_2)) <= eps).all())
# Result: False
```
Contributor guide
Research direction
Start by running the supplied Python reproduction with CatBoostRegressor, Pool, the Poisson loss function, baselines, and the four prediction cases. Compare default and RawFormulaVal predictions on train_x and tr_pool_2; done means the behavior matches the stated prediction relationships or their intended semantics are clarified.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100