automl / automl/Auto-PyTorch

[Bug] Cross validation yields unexpected additional info

Open
#349 0 comments 0 reactions 1 assignee Claimed by @ravinkohli View on GitHub
bug
Dominant language
Python
Stars
2.5k
Forks
300
PR merge metrics
No merged PRs in 30d

Description

# Issue Description
It seems abstract evaluator does not save cross-validated performance for opt/test datasets.
Also train_evaluator.py is too messy and requires major refactoring.

## Possible Solution
Modify `calculate_auxiliary_losses` in `abstract_evaluator.py`

## Your Code

When using the new results_manager, the following yields the issue.
(Although I fixed it)
```
python -m pytest test/test_api/test_api.py::test_tabular_classification[CrossValTypes.k_fold_cross_validation-resampling_strategy_args1-40981] -W ignore
```

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.