AnswerDotAI / AnswerDotAI/byaldi
qwen2 models causing indexerror
- Dominant language
- Python
- Stars
- 851
- Forks
- 91
- PR merge metrics
- No merged PRs in 30d
Description
I'm trying to run the example quick_overview.ipynb on m1 mbp.
```import os
from dotenv import load_dotenv
from pathlib import Path
from byaldi import RAGMultiModalModel
load_dotenv()
# print(os.environ["HF_TOKEN"])
# Initialize RAGMultiModalModel
model = RAGMultiModalModel.from_pretrained("vidore/colqwen2-v1.0", device="mps")
# Test indexing
metadata = [{"filename":file_name} for file_name in os.listdir("docs")]
index_name = "attention_index"
model.index(
input_path=Path("docs/"),
index_name=index_name,
store_collection_with_index=False,
metadata=metadata,
overwrite=True
)
# BLEU tables are on page 8 and 9. We've indexed the pdf and its evil mustached twin, so we should see similar scores occur twice for every relevant page.
query = "what's the BLEU score of this new strange method?"
results = model.search(query, k=5)
print(f"Search results for '{query}':")
for result in results:
print(f"Doc ID: {result.doc_id}, Page: {result.page_num}, Score: {result.score}")
print("Test completed successfully!")
```
But got the following error:
> ---------------------------------------------------------------------------
> IndexError Traceback (most recent call last)
> Cell In[6], [line 5](vscode-notebook-cell:?execution_count=6&line=5)
> [2](vscode-notebook-cell:?execution_count=6&line=2) metadata = [{"filename":file_name} for file_name in os.listdir("docs")]
> [4](vscode-notebook-cell:?execution_count=6&line=4) index_name = "attention_index"
> ----> [5](vscode-notebook-cell:?execution_count=6&line=5) model.index(
> [6](vscode-notebook-cell:?execution_count=6&line=6) input_path=Path("docs/"),
> [7](vscode-notebook-cell:?execution_count=6&line=7) index_name=index_name,
> [8](vscode-notebook-cell:?execution_count=6&line=8) store_collection_with_index=False,
> [9](vscode-notebook-cell:?execution_count=6&line=9) metadata=metadata,
> [10](vscode-notebook-cell:?execution_count=6&line=10) overwrite=True
> [11](vscode-notebook-cell:?execution_count=6&line=11) )
> [13](vscode-notebook-cell:?execution_count=6&line=13) # BLEU tables are on page 8 and 9. We've indexed the pdf and its evil mustached twin, so we should see similar scores occur twice for every relevant page.
> [14](vscode-notebook-cell:?execution_count=6&line=14) query = "what's the BLEU score of this new strange method?"
> ... ...
>
> File /opt/homebrew/Caskroom/miniforge/base/envs/colpali/lib/python3.12/site-packages/transformers/models/qwen2_vl/modeling_qwen2_vl.py:404, in VisionSdpaAttention.forward(self, hidden_states, cu_seqlens, rotary_pos_emb)
> [402](https://file+.vscode-resource.vscode-cdn.net/opt/homebrew/Caskroom/miniforge/base/envs/colpali/lib/python3.12/site-packages/transformers/models/qwen2_vl/modeling_qwen2_vl.py:402) k = k.transpose(0, 1)
> [403](https://file+.vscode-resource.vscode-cdn.net/opt/homebrew/Caskroom/miniforge/base/envs/colpali/lib/python3.12/site-packages/transformers/models/qwen2_vl/modeling_qwen2_vl.py:403) v = v.transpose(0, 1)
> --> [404](https://file+.vscode-resource.vscode-cdn.net/opt/homebrew/Caskroom/miniforge/base/envs/colpali/lib/python3.12/site-packages/transformers/models/qwen2_vl/modeling_qwen2_vl.py:404) attn_output = F.scaled_dot_product_attention(q, k, v, attention_mask, dropout_p=0.0)
> [405](https://file+.vscode-resource.vscode-cdn.net/opt/homebrew/Caskroom/miniforge/base/envs/colpali/lib/python3.12/site-packages/transformers/models/qwen2_vl/modeling_qwen2_vl.py:405) attn_output = attn_output.transpose(0, 1)
> [406](https://file+.vscode-resource.vscode-cdn.net/opt/homebrew/Caskroom/miniforge/base/envs/colpali/lib/python3.12/site-packages/transformers/models/qwen2_vl/modeling_qwen2_vl.py:406) attn_output = attn_output.reshape(seq_length, -1)
>
> IndexError: Dimension out of range (expected to be in range of [-3, 2], but got 3)
It seems the colpali models works fine, but qwen2 models cause the same error. Thanks for any help how to fix this.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.