terminated和done
- Dominant language
- Python
- Stars
- 3.4k
- Forks
- 312
- Avg merge
- 1h 2m
- Merged PRs (30d)
- 2
Description
按照gym.Env.step接口定义:
- terminated表示 s_t 是MDP终止状态(智能体成功、失败、死亡),s_t+1 不存在,计算TD-Error时不能再使用 Q_t+1 或 V_t+1
- truncated表示仿真被截断(达到最大仿真步或仿真异常),s_t+1 可能存在
- done = terminated or truncated,done时 s_t+1 也可能存在
但最新版roll/agentic/env中的几个环境实现直接令 terminated = done,但这两个参数含义通常情况下不等价。
另外,希望增加类似register(id, config_class, env_class)接口,方便将自定义环境注册到REGISTERED_ENVS和REGISTERED_ENV_CONFIGS中
Contributor guide
No contributing guide indexed for this repository
Research direction
Read the gym.Env.step contract first, then inspect the environment implementations under roll/agentic/env and the REGISTERED_ENVS and REGISTERED_ENV_CONFIGS structures. Done means terminated and truncated preserve their distinct meanings while done is their OR, and custom environment/configuration classes can be registered through a suitable interface.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100