A Bug In LossMetric which will lead to train.loss less then zero
- Lingua principale
- Java
- Stelle
- 6.8k
- Fork
- 1.6k
- Merge medio
- 44m
- PR unite (30g)
- 1
Descrizione
In LossMetric class, the sampleNum is Int and when sampleNum larger than 2^32, the sampleNum will overflow. the code is
class LossMetric(var sampleNum: Int) extends Metric {
var loss: Double = 0.0
override def merge(other: this.type): this.type = {
this.sampleNum += other.sampleNum
this.loss += other.loss
this
}
the phenomenon is
19/09/17 15:12:53 INFO client.AngelClient: Epoch: 1. Metrics={"train.loss":-17.665040,"validate.loss":0.313661}
19/09/17 15:31:06 INFO client.AngelClient: Epoch: 2. Metrics={"train.loss":-16.968140,"validate.loss":0.300109}
19/09/17 15:42:20 INFO client.AngelClient: Epoch: 3. Metrics={"train.loss":-16.734170,"validate.loss":0.293660}
19/09/17 15:51:09 INFO client.AngelClient: Epoch: 4. Metrics={"train.loss":-16.385350,"validate.loss":0.284851}
19/09/17 16:01:57 INFO client.AngelClient: Epoch: 5. Metrics={"train.loss":-16.013730,"validate.loss":0.278738}
19/09/17 16:26:41 INFO client.AngelClient: Epoch: 6. Metrics={"train.loss":-15.925190,"validate.loss":0.278801}
19/09/17 17:06:11 INFO client.AngelClient: Epoch: 7. Metrics={"train.loss":-15.980800,"validate.loss":0.280390}
19/09/17 17:18:53 INFO client.AngelClient: Epoch: 8. Metrics={"train.loss":-15.683660,"validate.loss":0.276296}
19/09/17 17:30:14 INFO client.AngelClient: Epoch: 9. Metrics={"train.loss":-15.226160,"validate.loss":0.269379}
19/09/17 17:42:57 INFO client.AngelClient: Epoch: 10. Metrics={"train.loss":-14.883470,"validate.loss":0.264469}
Guida per i contributori
Apri la guida per i contributori
Valutazione
Questa issue non è ancora stata valutata.