AI4Finance-Foundation / AI4Finance-Foundation/FinRL

Error training the agent for Paper_Trading

未關閉
#725 2 則留言 0 個 reaction 已指派 0 人 在 GitHub 檢視
bug
主要語言
Jupyter Notebook
星號
16.3k
分支
3.5k
PR 合併指標
30 天內沒有已合併 PR

描述

When using FinRL_PaperTrading_Demo on a brand new installation (Python 3.9.13) the notebook crashes at the training step.
```
train(start_date = '2021-10-11',
end_date = '2021-10-15',
ticker_list = ticker_list,
data_source = 'alpaca',
time_interval= candle_time_interval,
technical_indicator_list= INDICATORS,
drl_lib='elegantrl',
env=env,
model_name='ppo',
API_KEY = API_KEY,
API_SECRET = API_SECRET,
API_BASE_URL = API_BASE_URL,
erl_params=ERL_PARAMS,
cwd='./papertrading_erl', #current_working_dir
break_step=1e5)

```
the following error happens
```
Data clean finished!
Successfully transformed into array
| Arguments Remove cwd: ./papertrading_erl
################################################################################
ID Step maxR | avgR stdR avgS stdS | expR objC etc.
---------------------------------------------------------------------------
AssertionError Traceback (most recent call last)
Cell In [19], line 1
----> 1 train(start_date = '2021-10-11',
2 end_date = '2021-10-15',
3 ticker_list = ticker_list,
4 data_source = 'alpaca',
5 time_interval= candle_time_interval,
6 technical_indicator_list= INDICATORS,
7 drl_lib='elegantrl',
8 env=env,
9 model_name='ppo',
10 API_KEY = API_KEY,
11 API_SECRET = API_SECRET,
12 API_BASE_URL = API_BASE_URL,
13 erl_params=ERL_PARAMS,
14 cwd='./papertrading_erl', #current_working_dir
15 break_step=1e5)

File ~/miniforge3/envs/FinRL/lib/python3.9/site-packages/finrl/train.py:60, in train(start_date, end_date, ticker_list, data_source, time_interval, technical_indicator_list, drl_lib, env, model_name, if_vix, **kwargs)
53 agent = DRLAgent_erl(
54 env=env,
55 price_array=price_array,
56 tech_array=tech_array,
57 turbulence_array=turbulence_array,
58 )
59 model = agent.get_model(model_name, model_kwargs=erl_params)
---> 60 trained_model = agent.train_model(
61 model=model, cwd=cwd, total_timesteps=break_step
62 )
63 elif drl_lib == "rllib":
64 total_episodes = kwargs.get("total_episodes", 100)

File ~/miniforge3/envs/FinRL/lib/python3.9/site-packages/finrl/agents/elegantrl/models.py:84, in DRLAgent.train_model(self, model, cwd, total_timesteps)
82 model.cwd = cwd
83 model.break_step = total_timesteps
---> 84 train_and_evaluate(model)

File ~/miniforge3/envs/FinRL/lib/python3.9/site-packages/elegantrl/train/run.py:115, in train_and_evaluate(***failed resolving arguments***)
113 r_exp = trajectory[3].mean().item()
114 torch.set_grad_enabled(True)
--> 115 logging_tuple = agent.update_net(trajectory)
116 torch.set_grad_enabled(False)
117 (if_reach_goal, if_save) = evaluator.evaluate_save_and_plot(agent.act, steps, r_exp, logging_tuple)

File ~/miniforge3/envs/FinRL/lib/python3.9/site-packages/elegantrl/agents/AgentPPO.py:159, in AgentPPO.update_net(self, buffer)
156 obj_actors = 0.0
158 update_times = int(buffer_size * self.repeat_times / self.batch_size)
--> 159 assert update_times >= 1
160 for _ in range(update_times):
161 indices = torch.randint(buffer_size, size=(self.batch_size,), requires_grad=False)

AssertionError:
```

This happens without changing any parameter, just the plain notebook as downloaded from the site.
Not sure if the error is in ElegantRL or int he parameters passed to train the agent.
update_times is at 0 because buffer_size=32 self.repeat_times=1 and self.batch_size=2048 when the error happens

This can be reproduced by just running the notebook. Same thing happens on Mac M1 and on ubuntu (aarch64)

After further investigation the same issue is also present in other notebooks, for example FinRL_Compare_ElegantRL_RLlib_Stablebaseline3 exhibit the exact same error message, so I suspect it's related to ElegantRL somehow)

貢獻指南

這個儲存庫沒有索引到貢獻指南

評估

這個 Issue 還沒有評估資料。

把新 issue 寄到你的電子郵件信箱

精選適合新手參與的 GitHub issue 摘要。