stdlib-js / stdlib-js/stdlib

[BUG]: missing multiplication in `_modifiedHuberLoss` implementation in `ml/incr/binary-classification`

Open Beginner friendly
#13,111 5 comments 0 reactions 0 assignees View on GitHub
Bug Machine Learning
Dominant language
JavaScript
Stars
6k
Forks
1.3k
Avg merge
1d 3h
Merged PRs (30d)
611

Description

### Description

I was going through the wikipedia pages as well as other library implementations of loss functions to implement standalone loss functions and their gradients under `ml/base/loss/float64/*`.
I noticed a possible bug in `_modifiedHuberLoss` function implemented under `ml/incr/binary-classification`

Current implementation:
```javascript
setReadOnly( Model.prototype, '_modifiedHuberLoss', function modifiedHuber( x, y ) {
var eta;
var d;

eta = this[ this._learningRateMethod ]();
this._regularize( eta );

d = y * this._dot( x.data, x.strides[ 0 ], x.offset );
if ( d < -1.0 ) {
this._add( x, 4.0*eta*y );
} else {
this._add( x, eta*( y-(d*y) ) );
}
return this;
});
```
[Link](https://github.com/stdlib-js/stdlib/blob/8bf91614301abf9249ff3ea9379baa4bc6c450e0/lib/node_modules/%40stdlib/ml/incr/binary-classification/lib/model.js#L334-L348)

I believe in the `else` branch, it should be `this._add( x, eta*2*( y-(d*y) ) )` instead of `this._add( x, eta*( y-(d*y) ) )`. That is, it seems like it is missing a **multiplication with 2** here.

References:
[Wikipedia](https://en.wikipedia.org/wiki/Huber_loss#Variant_for_classification)
[sklearn](https://github.com/scikit-learn/scikit-learn/blob/fe2edb3cdbd75ae4e662fda67dcb19277258792b/sklearn/linear_model/_sgd_fast.pyx.tp#L145)

cc: @Planeshifter

### Related Issues

_No response_

### Questions

No.

### Demo

_No response_

### Reproduction

- a
- b
- c

### Expected Results

```math
\frac{\partial \ell}{\partial p} =
\begin{cases}
-4y & \text{if } yp < -1 \\
-2y(1 - yp) & \text{if } -1 \le yp \le 1 \\
0 & \text{if } yp > 1
\end{cases}
```

### Actual Results

```shell

```

### Version

_No response_

### Environments

N/A

### Browser Version

_No response_

### Node.js / npm Version

_No response_

### Platform

_No response_

### Checklist

- [x] Read and understood the [Code of Conduct](https://github.com/stdlib-js/stdlib/blob/develop/CODE_OF_CONDUCT.md).
- [x] Searched for existing issues and pull requests.

Contributor guide

Open the contributing guide

Research direction

Read lib/node_modules/@stdlib/ml/incr/binary-classification/lib/model.js, starting at Model.prototype._modifiedHuberLoss, and compare the else branch with the expected gradient and referenced implementations. Done means the implementation applies the proposed factor of 2 consistently with the stated derivative and preserves the existing behavior of the other branch.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript
Domain
machine-learning
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
72/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.