benchopt / benchopt/benchmark_lasso

BUG snapml early stopping with `gpu=True` on leukemia

Open
#105 5 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
14
Forks
24
PR merge metrics
No merged PRs in 30d

Description

`snapml` solver stops if the patience is not high enough on the `leukemia` dataset (but this seem to be highly dependent on the hardware - to confirm though).

A quick script to reproduce:
```python
from snapml import LinearRegression
import numpy as np

from sklearn.datasets import fetch_openml
from sklearn.preprocessing import LabelBinarizer

X, y = fetch_openml("leukemia", return_X_y=True)
X = X.to_numpy()
y = LabelBinarizer().fit_transform(y)[:, 0].astype(X.dtype)

lmbd = 0.01 * max(abs(X.T @ y))
clf = LinearRegression(
fit_intercept=False,
regularizer=lmbd,
penalty="l1",
tol=0,
dual=False,
use_gpu=True,
verbose=True,
generate_training_history="full",
)

clf.max_iter = 4
clf.fit(X, y)
print(np.where(clf.coef_.squeeze()))
print(clf.training_history_)
```

What we observed for now is that with GeForce RTX (2080 and 2090) SUPER (and two different cuda 11.X versions), the objective stays put and in practice in Benchopt we stop the run and do not converge.
BUT, with a QUADRO T2000, the objective decreases at the last `iter`, meaning in Benchopt we have a full curve showing the convergence.
poke @mathurinm for the original test script and issue

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with the reproduction script in the issue and run it on the reported GPU and CUDA combinations. Compare the objective values, final iteration, and training_history_ output, then investigate why convergence differs by hardware. Done means the hardware-dependent early-stopping behavior is reproduced and its cause is established.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, scikit-learn
Domain
machine-learning, performance
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.