benchopt / benchopt/benchmark_lasso
BUG snapml early stopping with `gpu=True` on leukemia
- Dominant language
- Python
- Stars
- 14
- Forks
- 24
- PR merge metrics
- No merged PRs in 30d
Description
`snapml` solver stops if the patience is not high enough on the `leukemia` dataset (but this seem to be highly dependent on the hardware - to confirm though).
A quick script to reproduce:
```python
from snapml import LinearRegression
import numpy as np
from sklearn.datasets import fetch_openml
from sklearn.preprocessing import LabelBinarizer
X, y = fetch_openml("leukemia", return_X_y=True)
X = X.to_numpy()
y = LabelBinarizer().fit_transform(y)[:, 0].astype(X.dtype)
lmbd = 0.01 * max(abs(X.T @ y))
clf = LinearRegression(
fit_intercept=False,
regularizer=lmbd,
penalty="l1",
tol=0,
dual=False,
use_gpu=True,
verbose=True,
generate_training_history="full",
)
clf.max_iter = 4
clf.fit(X, y)
print(np.where(clf.coef_.squeeze()))
print(clf.training_history_)
```
What we observed for now is that with GeForce RTX (2080 and 2090) SUPER (and two different cuda 11.X versions), the objective stays put and in practice in Benchopt we stop the run and do not converge.
BUT, with a QUADRO T2000, the objective decreases at the last `iter`, meaning in Benchopt we have a full curve showing the convergence.
poke @mathurinm for the original test script and issue
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with the reproduction script in the issue and run it on the reported GPU and CUDA combinations. Compare the objective values, final iteration, and training_history_ output, then investigate why convergence differs by hardware. Done means the hardware-dependent early-stopping behavior is reproduced and its cause is established.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, scikit-learn
- Domain
- machine-learning, performance
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100