Add regression tests on quality & performance
- 主要语言
- Python
- 星标
- 712
- 派生
- 96
- 平均合并
- 3 天 1 小时
- 30 天内合并 PR
- 10
描述
Aim is having control experiments with known result metrics for:
- autotesting regression in performance
- evaluating FEDOT after new features on these examples
First part of the issue is to write general AutoML tests (or adapt existing tests). Currently we have several integration tests that are close to such regression tests.
- [ ] Modify `test_real_cases.py` to use real expected metrics or baselines.
- [ ] Test AutoML quality on all kinds of tasks (classification, regression, forecasting)
- [x] Test on multi objective opt: must improve on all metrics -- done in `test_quality_improvement.py`
Second part of the issue is writing new tests for important features:
- [ ] Tuning: optimization with tuning must be better than just optimization)
- [ ] Preprocessing must handle some bad dirty dataset (maybe there is a test for that)
- [ ] Test that adaptive evo scheme converges faster than standard scheme (populational)
Third part is
- [ ] Test on comparisons with other AutoMLs -- FEDOT must return at least comparable quality or better (see and enhance `test_automl.py`)
This issue can be done in several PRs.
Regression tests must be put in `tests/intergration/`.
贡献指南
评估
这个 Issue 还没有评估数据。