AgentPPO中的explore_one_env

请教下AgentPPO中的explore_one_env中的splice list 的作用是什么

` def explore_one_env(self, env, target_step):
traj_temp = list()

state = self.states[0]
last_done = 0
for i in range(target_step):
action, noise = [ary[0] for ary in self.select_actions((state,))]
next_state, reward, done, _ = env.step(np.tanh(action))
traj_temp.append((state, reward, done, action, noise))
if done:
state = env.reset()
last_done = i
else:
state = next_state
self.states[0] = state

'''splice list'''
traj_list = self.traj_list[0] + traj_temp[:last_done + 1]
self.traj_list[0] = traj_temp[last_done:]
return traj_list`

Start with AgentPPO.explore_one_env and trace where self.traj_list is initialized and consumed. Follow traj_temp, last_done, and the concatenation and slices around the “splice list” comment. Done means documenting why completed-episode data is returned while the unfinished rollout remains buffered.

Escrito por el modelo de indexación a partir del texto del issue.