facebookresearch / facebookresearch/ImageBind

IMU Input Dimensions are Unclear - Missing Information on Data Prep

Open
#66 10 comments 5 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
9.1k
Forks
842
PR merge metrics
No merged PRs in 30d

Description

Hello,

What is the required format for IMU input embeddings? Or rather, why does T have to be 2000?
I've tried to run the code using sample embeddings as specified in the appendix of the paper.

> For IMU we use a 6×T tensor to represent the sequence of IMU sensor readings over time.

Initially I tried to use the sample from the Ego-4D dataset: https://ego4d-data.org/docs/data/imu/

but this kept throwing size mismatch errors

I am trying to create a joint embedding for a single
- RGB Image -> (1,3, 224, 224)
- Depth Image -> (1,1, 224, 224)
- IMU reading -> (1, 6, T) where T represents the number of time steps. I tried a few different options for T, but found **2000** to be the only one that doesn't throw a size mismatch and produces embeddings for each

Does this mean the model requires a minimum of 2000 time steps for IMU sensors?

Thank you for your help

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.