google-deepmind / google-deepmind/deepmind-research

[RL Unplugged] - Trained policies for finger_turn_hard do not match the datasets

Open
#465 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Jupyter Notebook
Stars
15.2k
Forks
2.9k
PR merge metrics
No merged PRs in 30d

Description

Hi, thanks for the good work.

I am running custom offline evaluation on the tasks from DeepMind Control Suite Dataset. I am taking the data from Tensorflow Datasets [here](https://www.tensorflow.org/datasets/catalog/rlu_control_suite) and downloading the maching policies from the GCP bucket [here](https://console.cloud.google.com/storage/browser/gresearch/deep-ope/rlunplugged?pageState=(%22StorageObjectListTable%22:(%22f%22:%22%255B%255D%22))&prefix=&forceOnObjectsSortingFiltering=false).

Most tasks work fine, but when I get to finger_turn_hard I get the following error while inferring the policies on the data:
![image](https://github.com/google-deepmind/deepmind-research/assets/74935134/7ab8a416-e879-45a0-afb0-5318e1ae5002)

It looks like the policy is expecting one of the keys to be "touch", however, this key is not in the observations. According to the documentation [here](https://www.tensorflow.org/datasets/catalog/rlu_control_suite#rlu_control_suitefinger_turn_hard), there is no dimension "touch" in this dataset.

For context, I am loading the policy using:
```python
tf.saved_model.load(policy_path)
```

And inferring using:
```python
if hasattr(policy, 'initial_state'):
action = policy(observation, ((),))[0]
else:
action = policy(observation)
```

Am I missing something here? How can I infer the finger_turn_hard policies on the provided datasets?

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.