alibaba / alibaba/ROLL

terminated和done

Open
#123 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
3.4k
Forks
312
Avg merge
1h 2m
Merged PRs (30d)
2

Description

按照gym.Env.step接口定义:

- terminated表示 s_t 是MDP终止状态(智能体成功、失败、死亡),s_t+1 不存在,计算TD-Error时不能再使用 Q_t+1 或 V_t+1

- truncated表示仿真被截断(达到最大仿真步或仿真异常),s_t+1 可能存在

- done = terminated or truncated,done时 s_t+1 也可能存在

但最新版roll/agentic/env中的几个环境实现直接令 terminated = done,但这两个参数含义通常情况下不等价。

另外,希望增加类似register(id, config_class, env_class)接口,方便将自定义环境注册到REGISTERED_ENVS和REGISTERED_ENV_CONFIGS中

Contributor guide

No contributing guide indexed for this repository

Research direction

Read the gym.Env.step contract first, then inspect the environment implementations under roll/agentic/env and the REGISTERED_ENVS and REGISTERED_ENV_CONFIGS structures. Done means terminated and truncated preserve their distinct meanings while done is their OR, and custom environment/configuration classes can be registered through a suitable interface.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.