[BUG] Chapter 15 naive prediction error with correction
- Langage dominant
- Jupyter Notebook
- Étoiles
- 30k
- Forks
- 13.1k
- Métriques de merge des PR
- Aucune PR mergée en 30 j
Description
The naive prediction in Chapter 15 reproduced below is incorrect. It also raises a visual red flag because the error from the network was about 10x smaller (about 0.027). Quoting from Chapter 15, https://github.com/ageron/handson-ml2/blob/master/15_processing_sequences_using_rnns_and_cnns.ipynb
```
Let's compare this performance with some baselines: naive predictions and a simple linear model:
In [31]:
Y_naive_pred = Y_valid[:, -1:]
np.mean(keras.metrics.mean_squared_error(Y_valid, Y_naive_pred))
Out[31]:
0.22278848
```
An accurate way to calculate the naive prediction error would be as follows. The naive error is actually smaller than the RNN walk forward error. This makes sense because it is actually very hard to beat the naive prediction, even when walking forward with real data as the notebook did. (For example, 70% of the naive error would be excellent. 10% of the naive error is ludicrous.)
```
actual_Y_naive=np.array([np.concatenate([x,y]) for x,y in zip(X_valid[:,-1],Y_valid[:,:-1])])
print(np.average(np.square(Y_valid-actual_Y_naive)))
np.mean(keras.metrics.mean_squared_error(Y_valid,actual_Y_naive))
0.01524985
0.015249849
```
The main problem with the code in the existing notebook is that it is not dimensionally consistent. Here are some accurate ways to calculate subsets of the naive error to make it easier to see.
This is just using the last validation value vs. the 2nd to last validation value.
```
Y_naive_pred = Y_valid[:, -2]
np.average(np.square(Y_valid[:,0]-X_valid[:,-1,0]))
np.mean(keras.metrics.mean_squared_error(Y_valid[:,-1], Y_naive_pred))
0.0152704455
0.0152704455
```
This is just using the first validation value vs. the last value in the input.
```
Y_naive_pred=X_valid[:,-1,0]
print(np.average(np.square(Y_valid[:,0]-X_valid[:,-1,0])))
np.mean(keras.metrics.mean_squared_error(Y_valid[:,0],Y_naive_pred))
0.015137789
0.015137789
```
Guide de contribution
Aucun guide de contribution indexé pour ce dépôt
Évaluation
Cette issue n'a pas encore été évaluée.