google-deepmind / google-deepmind/dm_control

How to create an environment to give depth images?

未关闭
#211 1 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看
主要语言
Python
星标
4.7k
派生
764
PR 合并指标
30 天内没有已合并 PR

描述

The vision environment provide only RGB images, instead of RGBD (depth) images, but I see some partial implementation of depth images. Here's my attempt to set `depth=True`, but it failes because the `dtype` is set to uint8 which cannot be `inf`.

```
import collections

import numpy as np
from dm_control import manipulation
from dm_control.manipulation.bricks import _reassemble
from dm_control.manipulation.shared import registry, tags, observations
from dm_control.manipulation.shared.observations import ObservationSettings, _ENABLED_FEATURE, _ENABLED_FTT, \
_DISABLED_FEATURE, CameraObservableSpec, ObservableSpec

class MyCameraObservableSpec(collections.namedtuple(
'CameraObservableSpec', ('depth', 'height', 'width') + ObservableSpec._fields)):
"""Configuration options for camera observables."""
__slots__ = ()

ENABLED_CAMERA_DEPTH = MyCameraObservableSpec(
height=84,
width=84,
enabled=True,
depth=True,
update_interval=1,
buffer_size=1,
delay=0,
aggregator=None,
corruptor=None)

VISION_DEPTH = ObservationSettings(
proprio=_ENABLED_FEATURE,
ftt=_ENABLED_FTT,
prop_pose=_DISABLED_FEATURE,
camera=ENABLED_CAMERA_DEPTH)

@registry.add(tags.VISION)
def my_env():
return _reassemble(obs_settings=VISION_DEPTH,
num_bricks=5,
randomize_initial_order=True,
randomize_desired_order=True)

env = manipulation.load('my_env', seed=0)

spec = env.action_spec()

for i in range(10):
a = np.random.uniform(spec.minimum, spec.maximum, spec.shape)
time_step = env.step(a)
```

贡献指南

打开贡献指南

调研方向

Start with dm_control.manipulation.shared.observations and the camera observable configuration used by _reassemble, then reproduce the depth=True failure in the provided environment example. Done means a vision environment can return depth observations alongside RGB data without the uint8/inf error.

由索引模型根据 Issue 内容生成。

评估

技术栈
numpy, python
领域
computer-vision
Issue 类型
功能
难度
3/5
预计耗时
1-2 天
活跃度
停滞
描述清晰度
需要澄清
新手友好度
25/100

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。