catboost / catboost/catboost

Learning rate doesnt take in account ignored features

Open
#2,843 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
9.1k
Forks
1.3k
PR merge metrics
No merged PRs in 30d

Description

### CatBoost Version: 1.2.7
### Environment: Google Colab

### Problem Description
When using pool and marking some features as ignored. These ignored features are still influencing learning rate. I guess.

```Python
features=list(X.columns[:100])
Small_Pool = Pool(data=X[features], label=y,has_header=True)
model2 = CatBoostClassifier(iterations=1000,random_seed=1)
model2.fit(Small_Pool,logging_level='Silent')
model2.learning_rate_
```
return 0.020483000203967094

```Python
features=list(X.columns[:100])
pool_features=Big_Pool.get_feature_names()
model = CatBoostClassifier(ignored_features=list(set(pool_features)-set(features)),iterations=1000,random_seed=1)
model.fit(Big_Pool,logging_level='Silent')
model.learning_rate_
```
return Learning rate 0.06321100145578384 instead of 0.020483000203967094

Bug is represented here:
https://colab.research.google.com/drive/16Buo-5yaeeq7qOZb6D6L8kjcnWXA0c4n?authuser=1#scrollTo=HJvc_BbjqJk4

Problem is fixed if i use model.drop_unused_features() but it is rather unobvious.

Contributor guide

Open the contributing guide

Research direction

Start by running the linked Google Colab reproduction with the two Pool configurations and compare their reported learning_rate_ values. Read the handling of ignored_features and drop_unused_features, then verify that equivalent selected features produce consistent learning rates without requiring drop_unused_features().

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.