stdlib-js / stdlib-js/stdlib

[BUG]: missing multiplication in `_modifiedHuberLoss` implementation in `ml/incr/binary-classification`

Aperta Adatta ai principianti
#13,111 5 commenti 0 reazioni 0 assegnatari Vedi su GitHub
Bug Machine Learning
Lingua principale
JavaScript
Stelle
6k
Fork
1.3k
Merge medio
1g 3h
PR unite (30g)
611

Descrizione

### Description

I was going through the wikipedia pages as well as other library implementations of loss functions to implement standalone loss functions and their gradients under `ml/base/loss/float64/*`.
I noticed a possible bug in `_modifiedHuberLoss` function implemented under `ml/incr/binary-classification`

Current implementation:
```javascript
setReadOnly( Model.prototype, '_modifiedHuberLoss', function modifiedHuber( x, y ) {
var eta;
var d;

eta = this[ this._learningRateMethod ]();
this._regularize( eta );

d = y * this._dot( x.data, x.strides[ 0 ], x.offset );
if ( d < -1.0 ) {
this._add( x, 4.0*eta*y );
} else {
this._add( x, eta*( y-(d*y) ) );
}
return this;
});
```
[Link](https://github.com/stdlib-js/stdlib/blob/8bf91614301abf9249ff3ea9379baa4bc6c450e0/lib/node_modules/%40stdlib/ml/incr/binary-classification/lib/model.js#L334-L348)

I believe in the `else` branch, it should be `this._add( x, eta*2*( y-(d*y) ) )` instead of `this._add( x, eta*( y-(d*y) ) )`. That is, it seems like it is missing a **multiplication with 2** here.

References:
[Wikipedia](https://en.wikipedia.org/wiki/Huber_loss#Variant_for_classification)
[sklearn](https://github.com/scikit-learn/scikit-learn/blob/fe2edb3cdbd75ae4e662fda67dcb19277258792b/sklearn/linear_model/_sgd_fast.pyx.tp#L145)

cc: @Planeshifter

### Related Issues

_No response_

### Questions

No.

### Demo

_No response_

### Reproduction

- a
- b
- c

### Expected Results

```math
\frac{\partial \ell}{\partial p} =
\begin{cases}
-4y & \text{if } yp < -1 \\
-2y(1 - yp) & \text{if } -1 \le yp \le 1 \\
0 & \text{if } yp > 1
\end{cases}
```

### Actual Results

```shell

```

### Version

_No response_

### Environments

N/A

### Browser Version

_No response_

### Node.js / npm Version

_No response_

### Platform

_No response_

### Checklist

- [x] Read and understood the [Code of Conduct](https://github.com/stdlib-js/stdlib/blob/develop/CODE_OF_CONDUCT.md).
- [x] Searched for existing issues and pull requests.

Guida per i contributori

Apri la guida per i contributori

Direzione di ricerca

Read lib/node_modules/@stdlib/ml/incr/binary-classification/lib/model.js, starting at Model.prototype._modifiedHuberLoss, and compare the else branch with the expected gradient and referenced implementations. Done means the implementation applies the proposed factor of 2 consistently with the stated derivative and preserves the existing behavior of the other branch.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Valutazione

Stack tecnologico
javascript
Ambito
machine-learning
Tipo di issue
Bug
Difficoltà
2/5
Tempo stimato
1-3 ore
Stato di attività
Tranquilla
Chiarezza
Specificata chiaramente
Idoneità per principianti
72/100

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.