When fitting a lasso, is normalization performed even if fit_intercept=False?
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 386
- Forks
- 36
- Avg merge
- 23h 24m
- Merged PRs (30d)
- 2
Description
I'm trying to understand what's the stopping criterion on the (sub)gradient norm, in order to scale gradient_tol correctly.
If I understood correctly, for a quadratic datatif/family="gaussian", and vanishing coefs, the grad argument of _norm_min_subgrad should be X.T @ y / len(y)
When I debug, I see this being true only when I have previously centered and normalized X. otherwise, even if in my GeneralizedLinearRegressor I have set fit_intercept=False, I see:
- a
P1array which is not constant - in
update_quadratics(), gradient_rows is equal toy / len(y)as expected, howevergrad = gradient_rows @ data.Xdoes not yield agradequal toX.T @ y / len(y).
It seems this is due toXbeing a
Mat: <class 'tabmat.dense_matrix.DenseMatrix'> of shape (10, 5). Shift: [0. 0. 0. 0. 0.] Mult: [1.03404146 1.22010398 0.81522932 1.0686763 1.27198591]so scaling happens.
Why is this the case eventhough clf.scale_predictors, and clf._center_predictors are False ?
To reproduce, print gradient_rows and grad in update_quadratics, and run:
from glum import GeneralizedLinearRegressor
import numpy as np
np.random.seed(0)
X = np.random.randn(10, 5)
y = np.random.randn(10)
X -= X.mean(axis=0)
alpha = 0.001
clf = GeneralizedLinearRegressor(
alpha=alpha, gradient_tol=1000, fit_intercept=False, family="gaussian",
l1_ratio=1, verbose=10).fit(X, y)
print("grad rows should be", y / len(y)) # it is
print("grad should be", X.T @ y / len(y)) # it is not!
If you center and scale X before, then the problem disappears
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with GeneralizedLinearRegressor.fit and the update_quadratics() path mentioned in the report, then inspect _norm_min_subgrad and tabmat's DenseMatrix shift and multiplier values. Run the provided reproduction and compare the transformed X with the expected X.T @ y / len(y). Done means establishing whether scaling is intended when fit_intercept=False and documenting or correcting the behavior with a regression test.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- numpy, python
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100