why action not any change after episode 3?

**Describe the bug**

i want to record training action logs, and check transaction process, but i found action logs(buy or sell) not any change after episode 3.

**To Reproduce**

in order to record action logs, i add a patch:

```
--- env_stocktrading.py 2024-01-16 10:10:50
+++ my_env_stocktrading.py 2024-01-16 10:11:03
@@ -74,6 +74,7 @@ class StockTradingEnv(gym.Env):
self.iteration = iteration
# initalize state
self.state = self._initiate_state()
+ self.action_logs = []

# initialize reward
self.reward = 0
\ No newline at end of file
@@ -117,6 +118,9 @@ class StockTradingEnv(gym.Env):
* sell_num_shares
* (1 - self.sell_cost_pct[index])
)
+ op_log = f"{self.data.iloc[index]['date']} sell {self.data.iloc[index]['tic']} {sell_num_shares} {self.data.iloc[index]['close']}, cost {sell_amount}"
+ self.action_logs.append(op_log)
+
# update balance
self.state[0] += sell_amount

\ No newline at end of file
@@ -187,6 +191,9 @@ class StockTradingEnv(gym.Env):
* buy_num_shares
* (1 + self.buy_cost_pct[index])
)
+ op_log = f"{self.data.iloc[index]['date']} buy {self.data.iloc[index]['tic']} {buy_num_shares} {self.data.iloc[index]['close']}, cost {buy_amount}"
+ self.action_logs.append(op_log)
+
self.state[0] -= buy_amount

self.state[index + self.stock_dim + 1] += buy_num_shares
\ No newline at end of file
@@ -220,6 +227,12 @@ class StockTradingEnv(gym.Env):
def step(self, actions):
self.terminal = self.day >= len(self.df.index.unique()) - 1
if self.terminal:
+ act_log_fn = f'./results/episode_{self.episode}_actions.log'
+ with open(act_log_fn, 'w') as f:
+ tmp = '\n'.join(self.action_logs)
+ f.write(tmp)
+ self.action_logs = []
+
# print(f"Episode: {self.episode}")
if self.make_plots:
self._make_plot()
\ No newline at end of file
```

check the files under ./results/episode_{self.episode}_actions.log after training is complete, i found agent actions not any changs after episode 3:

```
MD5 (episode_10_actions.log) = 9ef06f3bb23b879346aa8db04dff93a6
MD5 (episode_11_actions.log) = 9ef06f3bb23b879346aa8db04dff93a6
MD5 (episode_12_actions.log) = 9ef06f3bb23b879346aa8db04dff93a6
MD5 (episode_13_actions.log) = 9ef06f3bb23b879346aa8db04dff93a6
MD5 (episode_14_actions.log) = 9ef06f3bb23b879346aa8db04dff93a6
MD5 (episode_15_actions.log) = 9ef06f3bb23b879346aa8db04dff93a6
MD5 (episode_16_actions.log) = 9ef06f3bb23b879346aa8db04dff93a6
MD5 (episode_17_actions.log) = 9ef06f3bb23b879346aa8db04dff93a6
MD5 (episode_18_actions.log) = 9ef06f3bb23b879346aa8db04dff93a6
MD5 (episode_19_actions.log) = 9ef06f3bb23b879346aa8db04dff93a6
MD5 (episode_2_actions.log) = ff7ef83f4b43d9fe443bfea9da1961f3
MD5 (episode_3_actions.log) = 94ef84d7b36636a4a4284aa72708456f
MD5 (episode_4_actions.log) = 9ef06f3bb23b879346aa8db04dff93a6
MD5 (episode_5_actions.log) = 9ef06f3bb23b879346aa8db04dff93a6
MD5 (episode_6_actions.log) = 9ef06f3bb23b879346aa8db04dff93a6
MD5 (episode_7_actions.log) = 9ef06f3bb23b879346aa8db04dff93a6
MD5 (episode_8_actions.log) = 9ef06f3bb23b879346aa8db04dff93a6
MD5 (episode_9_actions.log) = 9ef06f3bb23b879346aa8db04dff93a6
```

**Expected behavior**

The agent should continuously learn during the training process and generate different transaction logs.