facebookresearch / facebookresearch/ImageBind
IMU Input Dimensions are Unclear - Missing Information on Data Prep
- Dominant language
- Python
- Stars
- 9.1k
- Forks
- 842
- PR merge metrics
- No merged PRs in 30d
Description
Hello,
What is the required format for IMU input embeddings? Or rather, why does T have to be 2000?
I've tried to run the code using sample embeddings as specified in the appendix of the paper.
> For IMU we use a 6×T tensor to represent the sequence of IMU sensor readings over time.
Initially I tried to use the sample from the Ego-4D dataset: https://ego4d-data.org/docs/data/imu/
but this kept throwing size mismatch errors
I am trying to create a joint embedding for a single
- RGB Image -> (1,3, 224, 224)
- Depth Image -> (1,1, 224, 224)
- IMU reading -> (1, 6, T) where T represents the number of time steps. I tried a few different options for T, but found **2000** to be the only one that doesn't throw a size mismatch and produces embeddings for each
Does this mean the model requires a minimum of 2000 time steps for IMU sensors?
Thank you for your help
Contributor guide
Assessment
This issue has not been assessed yet.