[BUG] Chapter 15 naive prediction error with correction
- Lenguaje dominante
- Jupyter Notebook
- Estrellas
- 30k
- Forks
- 13.1k
- Métricas de merge de PR
- Sin PR fusionados en 30 d
Descripción
The naive prediction in Chapter 15 reproduced below is incorrect. It also raises a visual red flag because the error from the network was about 10x smaller (about 0.027). Quoting from Chapter 15, https://github.com/ageron/handson-ml2/blob/master/15_processing_sequences_using_rnns_and_cnns.ipynb
```
Let's compare this performance with some baselines: naive predictions and a simple linear model:
In [31]:
Y_naive_pred = Y_valid[:, -1:]
np.mean(keras.metrics.mean_squared_error(Y_valid, Y_naive_pred))
Out[31]:
0.22278848
```
An accurate way to calculate the naive prediction error would be as follows. The naive error is actually smaller than the RNN walk forward error. This makes sense because it is actually very hard to beat the naive prediction, even when walking forward with real data as the notebook did. (For example, 70% of the naive error would be excellent. 10% of the naive error is ludicrous.)
```
actual_Y_naive=np.array([np.concatenate([x,y]) for x,y in zip(X_valid[:,-1],Y_valid[:,:-1])])
print(np.average(np.square(Y_valid-actual_Y_naive)))
np.mean(keras.metrics.mean_squared_error(Y_valid,actual_Y_naive))
0.01524985
0.015249849
```
The main problem with the code in the existing notebook is that it is not dimensionally consistent. Here are some accurate ways to calculate subsets of the naive error to make it easier to see.
This is just using the last validation value vs. the 2nd to last validation value.
```
Y_naive_pred = Y_valid[:, -2]
np.average(np.square(Y_valid[:,0]-X_valid[:,-1,0]))
np.mean(keras.metrics.mean_squared_error(Y_valid[:,-1], Y_naive_pred))
0.0152704455
0.0152704455
```
This is just using the first validation value vs. the last value in the input.
```
Y_naive_pred=X_valid[:,-1,0]
print(np.average(np.square(Y_valid[:,0]-X_valid[:,-1,0])))
np.mean(keras.metrics.mean_squared_error(Y_valid[:,0],Y_naive_pred))
0.015137789
0.015137789
```
Guía de contribución
No hay ninguna guía de contribución indexada para este repositorio
Evaluación
Este issue todavía no se ha evaluado.