Cadene / Cadene/vqa.pytorch

What should be the size of the input_question in engine.py ?

Open
#53 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
733
Forks
176
PR merge metrics
No merged PRs in 30d

Description

Hi, thank you so much for providing your code!
I want to check the output shapes at every stage of the model. I plan to do that by passing in some random tensors of the specific shape required model as I really cannot download and extract the entire VQA dataset just for testing.

Can anyone please let me know the shape of `input_question` in the `engine.py` file [here](https://github.com/Cadene/vqa.pytorch/blob/69d7a43fb02c0332176915a4a23fc47cab08b1d2/vqa/lib/engine.py#L19) (with and without attention) before passing it as input to the model and how to create a random tensor of that specific shape. I tried using
```
word_to_ix = {"hello": 0, "world": 1}
lookup_tensor = torch.tensor([word_to_ix["world"]], dtype=torch.long).cuda()

input_img = torch.randn(1,2048,14,14)
input_img = x1.long()
input_img = Variable(torch.LongTensor(input_img)).cuda()

out = model(input_img,lookup_tensor)
```

But I get this error
```
ensor([1], device='cuda:0')
Traceback (most recent call last):
File "test.py", line 83, in
out = model(x1,lookup_tensor)
File "/home/sarvani/anaconda3/envs/MMTOD_env/lib/python3.7/site-packages/torch/nn/modules/module.py", line 532, in __call__
result = self.forward(*input, **kwargs)
File "/home/sarvani/Desktop/SaiCharan/misc/vqa.pytorch/vqa/models/att.py", line 160, in forward
x_q_vec = self.seq2vec(input_q)
File "/home/sarvani/anaconda3/envs/MMTOD_env/lib/python3.7/site-packages/torch/nn/modules/module.py", line 532, in __call__
result = self.forward(*input, **kwargs)
File "/home/sarvani/Desktop/SaiCharan/misc/vqa.pytorch/vqa/models/seq2vec.py", line 62, in forward
lengths = process_lengths(input)
File "/home/sarvani/Desktop/SaiCharan/misc/vqa.pytorch/vqa/models/seq2vec.py", line 12, in process_lengths
max_length = input.size(1)
IndexError: Dimension out of range (expected to be in range of [-1, 0], but got 1)

```
Any help would be appreciated. Thanks!

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with engine.py and trace the model call into att.py and models/seq2vec.py, especially process_lengths in seq2vec.py. Compare the expected question tensor dimensions with the failing one-dimensional lookup_tensor and the input image example. Done means the model accepts representative tensors without the reported dimension error and the intermediate shapes can be inspected.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, pytorch
Domain
machine-learning
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.