ageron / ageron/handson-ml2

[BUG] Chapter 15 naive prediction error with correction

Ouverte
#421 5 commentaires 0 réactions 0 personnes assignées Voir sur GitHub
Langage dominant
Jupyter Notebook
Étoiles
30k
Forks
13.1k
Métriques de merge des PR
Aucune PR mergée en 30 j

Description

The naive prediction in Chapter 15 reproduced below is incorrect. It also raises a visual red flag because the error from the network was about 10x smaller (about 0.027). Quoting from Chapter 15, https://github.com/ageron/handson-ml2/blob/master/15_processing_sequences_using_rnns_and_cnns.ipynb
```
Let's compare this performance with some baselines: naive predictions and a simple linear model:

In [31]:
Y_naive_pred = Y_valid[:, -1:]
np.mean(keras.metrics.mean_squared_error(Y_valid, Y_naive_pred))
Out[31]:
0.22278848
```

An accurate way to calculate the naive prediction error would be as follows. The naive error is actually smaller than the RNN walk forward error. This makes sense because it is actually very hard to beat the naive prediction, even when walking forward with real data as the notebook did. (For example, 70% of the naive error would be excellent. 10% of the naive error is ludicrous.)
```
actual_Y_naive=np.array([np.concatenate([x,y]) for x,y in zip(X_valid[:,-1],Y_valid[:,:-1])])
print(np.average(np.square(Y_valid-actual_Y_naive)))
np.mean(keras.metrics.mean_squared_error(Y_valid,actual_Y_naive))
0.01524985
0.015249849
```

The main problem with the code in the existing notebook is that it is not dimensionally consistent. Here are some accurate ways to calculate subsets of the naive error to make it easier to see.

This is just using the last validation value vs. the 2nd to last validation value.
```
Y_naive_pred = Y_valid[:, -2]
np.average(np.square(Y_valid[:,0]-X_valid[:,-1,0]))
np.mean(keras.metrics.mean_squared_error(Y_valid[:,-1], Y_naive_pred))
0.0152704455
0.0152704455
```

This is just using the first validation value vs. the last value in the input.
```
Y_naive_pred=X_valid[:,-1,0]
print(np.average(np.square(Y_valid[:,0]-X_valid[:,-1,0])))
np.mean(keras.metrics.mean_squared_error(Y_valid[:,0],Y_naive_pred))
0.015137789
0.015137789
```

Guide de contribution

Aucun guide de contribution indexé pour ce dépôt

Évaluation

Cette issue n'a pas encore été évaluée.

Recevez les nouvelles issues par e-mail

Un résumé court des issues GitHub adaptées aux débutants.