google-deepmind / google-deepmind/dm_control

How to create an environment to give depth images?

Open
#211 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
4.7k
Forks
764
PR merge metrics
No merged PRs in 30d

Description

The vision environment provide only RGB images, instead of RGBD (depth) images, but I see some partial implementation of depth images. Here's my attempt to set `depth=True`, but it failes because the `dtype` is set to uint8 which cannot be `inf`.

```
import collections

import numpy as np
from dm_control import manipulation
from dm_control.manipulation.bricks import _reassemble
from dm_control.manipulation.shared import registry, tags, observations
from dm_control.manipulation.shared.observations import ObservationSettings, _ENABLED_FEATURE, _ENABLED_FTT, \
_DISABLED_FEATURE, CameraObservableSpec, ObservableSpec

class MyCameraObservableSpec(collections.namedtuple(
'CameraObservableSpec', ('depth', 'height', 'width') + ObservableSpec._fields)):
"""Configuration options for camera observables."""
__slots__ = ()

ENABLED_CAMERA_DEPTH = MyCameraObservableSpec(
height=84,
width=84,
enabled=True,
depth=True,
update_interval=1,
buffer_size=1,
delay=0,
aggregator=None,
corruptor=None)

VISION_DEPTH = ObservationSettings(
proprio=_ENABLED_FEATURE,
ftt=_ENABLED_FTT,
prop_pose=_DISABLED_FEATURE,
camera=ENABLED_CAMERA_DEPTH)

@registry.add(tags.VISION)
def my_env():
return _reassemble(obs_settings=VISION_DEPTH,
num_bricks=5,
randomize_initial_order=True,
randomize_desired_order=True)

env = manipulation.load('my_env', seed=0)

spec = env.action_spec()

for i in range(10):
a = np.random.uniform(spec.minimum, spec.maximum, spec.shape)
time_step = env.step(a)
```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.