google-deepmind / google-deepmind/open_x_embodiment
Saytap dataset all image/wrist_image are 0s
- Dominant language
- Jupyter Notebook
- Stars
- 2k
- Forks
- 127
- PR merge metrics
- No merged PRs in 30d
Description
## utokyo_saytap_converted_externally_to_rlds
At version 0.1.0 all `image` in `utokyo_saytap_converted_externally_to_rlds` as well as `wrist_image` are empty frames (all 0s)

```
gs://gresearch/robotics/utokyo_saytap_converted_externally_to_rlds/0.1.0
```
## Steps to reproduce
```
pip install rerun-sdk
pip install rerun-notebook
```
```
dataset = 'utokyo_saytap_converted_externally_to_rlds'
display_key = 'image'
b = tfds.builder_from_directory(builder_dir=dataset2path(dataset))
ds = b.as_dataset(split='train[:10]').shuffle(10) # take only first 10 episodes
b.info.features['steps']
episode = next(iter(ds))
images = [step['observation']['image'] for step in episode['steps']]
actions = {
# 'rotation_delta': [step['action']['rotation_delta'] for step in episode['steps']],
'action': [step['action'] for step in episode['steps']]
}
pil_images = [Image.fromarray(image.numpy()) for image in images]
```
```
rr.init("Open Embodiment X")
timesteps = len(pil_images)
for tidx in range(timesteps):
rr.set_time_sequence("frame_idx", tidx)
rr.set_time_seconds("sensor_time", 0.33 * tidx)
rr.log("input", rr.Image(np.array(pil_images[tidx])))
for action_name, scalers in actions.items():
for tidx, scaler in enumerate(scalers):
rr.set_time_sequence("frame_idx", tidx)
for idx, val in enumerate(scaler):
rr.log(
f"action/{action_name}/{idx}",
rr.Scalar(val),
)
rr.notebook_show(width=1200, height=800)
```
Here's a rerun.io [saytap-episode.rrd.zip](https://github.com/user-attachments/files/16443035/saytap-episode.rrd.zip) which you play here — https://rerun.io/viewer
Contributor guide
Assessment
This issue has not been assessed yet.