AI4Finance-Foundation / AI4Finance-Foundation/FinRL
Full dataset normalization biases test set
- Linguagem predominante
- Jupyter Notebook
- Estrelas
- 16.3k
- Forks
- 3.5k
- Métricas de merge de PRs
- Nenhum PR com merge em 30d
Descrição
In [FinRL_PortfolioOptimizationEnv_Demo.ipynb](https://github.com/AI4Finance-Foundation/FinRL/blob/master/examples/FinRL_PortfolioOptimizationEnv_Demo.ipynb) all data used for training and testing are normalised at once:
```
portfolio_norm_df = GroupByScaler(by="tic", scaler=MaxAbsScaler).fit_transform(portfolio_raw_df)
```
By doing so we tell the model when across all our observations (training + test) the stock price of a given position in the portfolio has been the highest.
@C4i0kun should we not instead normalize only the batch of observations for a given time step? Avoiding to introduce forward looking information, that is supposed to be unknown.
For example, we could delegate the normalisation to env_portfolio_optimization.py:
```
# define data to be used in this time step
self._data_raw = self._df[
(self._df[self._time_column] >= start_time)
& (self._df[self._time_column] <= end_time)
][[self._time_column, self._tic_column] + self._features]
# normalise observations
self._data = GroupByScaler(by="tic", scaler=MaxAbsScaler, columns=['close','high','low']).fit_transform(self._data_raw)
```
Guia de contribuição
Nenhum guia de contribuição indexado para este repositório
Avaliação
Esta issue ainda não foi avaliada.