google-deepmind / google-deepmind/deepmind-research

Bug in dm_control_suite_d4pg.ipynb

Open
#100 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Jupyter Notebook
Stars
15.2k
Forks
2.9k
PR merge metrics
No merged PRs in 30d

Description

The example code provided for running D4PG with tasks from DeepMind Control Suite has a bug.

The dataset preprocess function should concatenate the observations/next observations dict so that a complete observation is passed to the agent.
```

def flatten_observation(observation):
keys = sorted(six.iterkeys(observation))
observation_arrays = [tf.reshape(observation[key], [-1]) for key in keys]
return tf.concat(observation_arrays, 0)

def preprocess_fn(sample):
o_tm1, a_tm1, r_t, d_t, o_t = sample.data[:5]
o_tm1 = flatten_observation(o_tm1)
o_t = flatten_observation(o_t)
return replay_sample.ReplaySample(
info=sample.info, data=(o_tm1, a_tm1, r_t, d_t, o_t))

def main(argv):
task = dm_control_suite.ControlSuite("cartpole_swingup")

environment = task.environment
environment_spec = specs.make_environment_spec(environment)

dataset = dm_control_suite.dataset(
"./tmp",
data_path=task.data_path,
shapes=task.shapes,
uint8_features=task.uint8_features,
num_threads=1,
batch_size=256,
num_shards=100)

dataset = dataset.map(preprocess_fn).batch(256)

```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.