Chapter 4: Ridge regression using Gradient Descent [QUESTION]
- Dominant language
- Jupyter Notebook
- Stars
- 30k
- Forks
- 13.1k
- PR merge metrics
- No merged PRs in 30d
Description
**Ridge regression using Gradient Descent**
Hi!
I was trying to implement a Ridge regression in gradient descent by adding alpha*theta to the MSE gradient vector (where theta is the parameter vector).
So I've used the following code:
```python
eta = 0.1 #learning rate
n_iterations = 1000
m=76
alpha = 1
theta_ridge = np.random.randn(2,1) #random starting theta
for iteration in range(n_iterations):
gradients = 2/m*X_b.T.dot(X_b.dot(theta_ridge)-y) + alpha*theta_ridge[1] #theta_ridge[1] in order to exclude the intercept (theta_ridge[0])
theta_ridge = theta_ridge - eta*gradients
theta_ridge
>> array([[-0.09590297],
[ 0.19180594]])
```
**Expected behavior**
I would expect the final parameters vector to be the same as the one that I find by using the Ridge module from sklearn.linear_model, but it is not:
```python
from sklearn.linear_model import Ridge
ridge_reg = Ridge(alpha=1, solver='lsqr')
ridge_reg.fit(X_b, y)
ridge_reg.coef_
>> array([[0. , 0.28397242]])
```
I've also tried different solvers (even Cholesky as reported in the book) but I always get different results.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.