benchopt / benchopt/benchopt

Unexpected stop because of check_convergence in stopping_criterion

Open
#533 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
326
Forks
74
Avg merge
47m
Merged PRs (30d)
1

Description

When running benchmarks with multiple solvers and the params max_runs set, some solvers will end up producing a much lower number of iterates than max_runs because `check_convergence` makes them exit before their time. This might not be intended behavior for some users. Especially for DL people using mini-batch SGD like solvers where it often happens that the objective (test error) starts rising again before going down because of non-convexity (finding local minima then "escaping" making objective rise to find a better one afterwards) , variance or double-descent phenomenon
A couple of things that would help:
- make it clearer that it stopped because of `check_convergence` in the logs
- allow users to disable this check
- implement more robust logic
- explain more about what this check is doing in the docs (notably the role of patience and tolerance (EPS) and their default values)

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.