AI4Finance-Foundation / AI4Finance-Foundation/FinRL

Error training the agent for Paper_Trading

Open
#725 2 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
Jupyter Notebook
Stars
16.3k
Forks
3.5k
PR merge metrics
No merged PRs in 30d

Description

When using FinRL_PaperTrading_Demo on a brand new installation (Python 3.9.13) the notebook crashes at the training step.
```
train(start_date = '2021-10-11',
end_date = '2021-10-15',
ticker_list = ticker_list,
data_source = 'alpaca',
time_interval= candle_time_interval,
technical_indicator_list= INDICATORS,
drl_lib='elegantrl',
env=env,
model_name='ppo',
API_KEY = API_KEY,
API_SECRET = API_SECRET,
API_BASE_URL = API_BASE_URL,
erl_params=ERL_PARAMS,
cwd='./papertrading_erl', #current_working_dir
break_step=1e5)

```
the following error happens
```
Data clean finished!
Successfully transformed into array
| Arguments Remove cwd: ./papertrading_erl
################################################################################
ID Step maxR | avgR stdR avgS stdS | expR objC etc.
---------------------------------------------------------------------------
AssertionError Traceback (most recent call last)
Cell In [19], line 1
----> 1 train(start_date = '2021-10-11',
2 end_date = '2021-10-15',
3 ticker_list = ticker_list,
4 data_source = 'alpaca',
5 time_interval= candle_time_interval,
6 technical_indicator_list= INDICATORS,
7 drl_lib='elegantrl',
8 env=env,
9 model_name='ppo',
10 API_KEY = API_KEY,
11 API_SECRET = API_SECRET,
12 API_BASE_URL = API_BASE_URL,
13 erl_params=ERL_PARAMS,
14 cwd='./papertrading_erl', #current_working_dir
15 break_step=1e5)

File ~/miniforge3/envs/FinRL/lib/python3.9/site-packages/finrl/train.py:60, in train(start_date, end_date, ticker_list, data_source, time_interval, technical_indicator_list, drl_lib, env, model_name, if_vix, **kwargs)
53 agent = DRLAgent_erl(
54 env=env,
55 price_array=price_array,
56 tech_array=tech_array,
57 turbulence_array=turbulence_array,
58 )
59 model = agent.get_model(model_name, model_kwargs=erl_params)
---> 60 trained_model = agent.train_model(
61 model=model, cwd=cwd, total_timesteps=break_step
62 )
63 elif drl_lib == "rllib":
64 total_episodes = kwargs.get("total_episodes", 100)

File ~/miniforge3/envs/FinRL/lib/python3.9/site-packages/finrl/agents/elegantrl/models.py:84, in DRLAgent.train_model(self, model, cwd, total_timesteps)
82 model.cwd = cwd
83 model.break_step = total_timesteps
---> 84 train_and_evaluate(model)

File ~/miniforge3/envs/FinRL/lib/python3.9/site-packages/elegantrl/train/run.py:115, in train_and_evaluate(***failed resolving arguments***)
113 r_exp = trajectory[3].mean().item()
114 torch.set_grad_enabled(True)
--> 115 logging_tuple = agent.update_net(trajectory)
116 torch.set_grad_enabled(False)
117 (if_reach_goal, if_save) = evaluator.evaluate_save_and_plot(agent.act, steps, r_exp, logging_tuple)

File ~/miniforge3/envs/FinRL/lib/python3.9/site-packages/elegantrl/agents/AgentPPO.py:159, in AgentPPO.update_net(self, buffer)
156 obj_actors = 0.0
158 update_times = int(buffer_size * self.repeat_times / self.batch_size)
--> 159 assert update_times >= 1
160 for _ in range(update_times):
161 indices = torch.randint(buffer_size, size=(self.batch_size,), requires_grad=False)

AssertionError:
```

This happens without changing any parameter, just the plain notebook as downloaded from the site.
Not sure if the error is in ElegantRL or int he parameters passed to train the agent.
update_times is at 0 because buffer_size=32 self.repeat_times=1 and self.batch_size=2048 when the error happens

This can be reproduced by just running the notebook. Same thing happens on Mac M1 and on ubuntu (aarch64)

After further investigation the same issue is also present in other notebooks, for example FinRL_Compare_ElegantRL_RLlib_Stablebaseline3 exhibit the exact same error message, so I suspect it's related to ElegantRL somehow)

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.