catboost / catboost/catboost

[Feature request] Propagate python's exception in callback

Open
#1,744 3 comments 3 reactions 0 assignees View on GitHub
python
Dominant language
C++
Stars
9.1k
Forks
1.3k
PR merge metrics
No merged PRs in 30d

Description

Thank you for providing callback functionality in 0.26.0! When I use this functionality from another library, I would like to raise an [exception](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.exceptions.TrialPruned.html) in `after_iteration`. For example, like [this](https://github.com/nzw0301/optuna/blob/d393b24e14f25b08dc2a98c484c8dc4447891b58/optuna/integration/catboost.py#L68).

In my understanding, unfortunately, we cannot do that since the callback function is called by cython's `cref` function of catboost. Thus my feature request is that it enables us to raise the python's exception in `after_iteration`. The following code is a concrete example of my problem.

## Reproducible procedure and codes

```bash
git clone git@github.com:nzw0301/optuna.git
cd optuna
git checkout catboost-example-case
python setup.py install # install optuna
pip install -U catboost # install catboost

python catboost_integration.py
```

```bash
Traceback (most recent call last):
File "/Users/nzw/Downloads/optuna/optuna/_optimize.py", line 216, in _run_trial
value_or_values = func(trial)
File "catboost_integration.py", line 47, in objective
gbm.fit(
File "/opt/homebrew/Caskroom/miniconda/base/envs/optuna/lib/python3.8/site-packages/catboost/core.py", line 4673, in fit
self._fit(X, y, cat_features, text_features, embedding_features, None, sample_weight, None, None, None, None, baseline, use_best_model,
File "/opt/homebrew/Caskroom/miniconda/base/envs/optuna/lib/python3.8/site-packages/catboost/core.py", line 1994, in _fit
self._train(
File "/opt/homebrew/Caskroom/miniconda/base/envs/optuna/lib/python3.8/site-packages/catboost/core.py", line 1425, in _train
self._object._train(train_pool, test_pool, params, allow_clear_pool, init_model._object if init_model else None)
SystemError: returned a result with an error set
Traceback (most recent call last):
File "_catboost.pyx", line 1161, in _catboost._CallbackAfterIteration
File "/opt/homebrew/Caskroom/miniconda/base/envs/optuna/lib/python3.8/site-packages/catboost/core.py", line 6423, in after_iteration
if not cb.after_iteration(info):
File "/Users/nzw/Downloads/optuna/optuna/integration/catboost.py", line 68, in after_iteration
raise optuna.TrialPruned(message)
optuna.exceptions.TrialPruned: Trial was pruned at iteration 5.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "_catboost.pyx", line 1155, in _catboost._CallbackAfterIteration
SystemError: returned a result with an error set

... # same error message above

Traceback (most recent call last):
File "catboost_integration.py", line 68, in
study.optimize(objective, n_trials=100, timeout=600)
File "/Users/nzw/Downloads/optuna/optuna/study.py", line 401, in optimize
_optimize(
File "/Users/nzw/Downloads/optuna/optuna/_optimize.py", line 65, in _optimize
_optimize_sequential(
File "/Users/nzw/Downloads/optuna/optuna/_optimize.py", line 162, in _optimize_sequential
trial = _run_trial(study, func, catch)
File "/Users/nzw/Downloads/optuna/optuna/_optimize.py", line 267, in _run_trial
raise func_err
File "/Users/nzw/Downloads/optuna/optuna/_optimize.py", line 216, in _run_trial
value_or_values = func(trial)
File "catboost_integration.py", line 47, in objective
gbm.fit(
File "/opt/homebrew/Caskroom/miniconda/base/envs/optuna/lib/python3.8/site-packages/catboost/core.py", line 4673, in fit
self._fit(X, y, cat_features, text_features, embedding_features, None, sample_weight, None, None, None, None, baseline, use_best_model,
File "/opt/homebrew/Caskroom/miniconda/base/envs/optuna/lib/python3.8/site-packages/catboost/core.py", line 1994, in _fit
self._train(
File "/opt/homebrew/Caskroom/miniconda/base/envs/optuna/lib/python3.8/site-packages/catboost/core.py", line 1425, in _train
self._object._train(train_pool, test_pool, params, allow_clear_pool, init_model._object if init_model else None)
SystemError: returned a result with an error set
```

## Dependencies

catboost version: 0.26.0 and 0.26.1
Operating System: MacOS
CPU: Apple M1 chip

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.