benchopt / benchopt/benchmark_logreg_l2

Make stochastic solvers fit in this benchmark

Open
#16 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
5
Forks
15
PR merge metrics
No merged PRs in 30d

Description

Now that we use `SufficientProcessCriterion` to stop the benchmark and that we can have multiple loss at once, there is no real gain to have separate benchmarks for `logreg_l2` with batch and stochastic solvers. We should thus merge benchopt/benchmark_stochastic_logreg_l2 and this benchmark. To do this, we should:

- [ ] port `SGD` solver in this benchmark
- [ ] add a notion of `train/test` losses in `objective.py`
- [ ] improve the plotting utils for multi value loss functions to get train/test graphs.
- [ ] close the benchopt/benchmark_stochastic_logreg_l2 repo

Moreover, to make this benchmark more inline with literature practices, we should try and reproduce the following figures:

* From [Schmidtt et al. 2014](https://arxiv.org/pdf/1309.2388.pdf)
![image](https://user-images.githubusercontent.com/3321081/132006409-8ef88dc1-fffb-4cfe-87de-27685aaa5bca.png)

* From [Leblond et al. 2017](http://proceedings.mlr.press/v54/leblond17a/leblond17a.pdf)
![image](https://user-images.githubusercontent.com/3321081/132006531-9a61a576-4476-43a7-927e-66ae9282973c.png)

Contributor guide

No contributing guide indexed for this repository

Research direction

Compare benchopt/benchmark_stochastic_logreg_l2 with this benchmark first, then inspect objective.py and the plotting utils. Port the SGD solver, represent train/test losses, produce train/test graphs, reproduce the referenced figures, and close the separate stochastic benchmark repository when the merged benchmark is complete.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.