Model does not support Flash Attention 2.0 yet

Running the `gradio_web_server_adhoc.py` or inference script with different models always leads to:

The error:
``` File "/home/dominik/Documents/repos/VideoLLaMA2/own_scripts/inference.py", line 33, in
inference()
File "/home/dominik/Documents/repos/VideoLLaMA2/own_scripts/inference.py", line 27, in inference
model, processor, tokenizer = model_init(model_path)
File "/home/dominik/Documents/repos/VideoLLaMA2/videollama2/__init__.py", line 17, in model_init
tokenizer, model, processor, context_len = load_pretrained_model(model_path, None, model_name, **kwargs)
File "/home/dominik/Documents/repos/VideoLLaMA2/videollama2/model/__init__.py", line 174, in load_pretrained_model
model = Videollama2Qwen2ForCausalLM.from_pretrained(model_path, low_cpu_mem_usage=True, config=config, **kwargs)
File "/home/dominik/miniconda3/envs/vllama/lib/python3.9/site-packages/transformers/modeling_utils.py", line 3550, in from_pretrained
model = cls(config, *model_args, **model_kwargs)
File "/home/dominik/Documents/repos/VideoLLaMA2/videollama2/model/videollama2_qwen2.py", line 50, in __init__
self.model = Videollama2Qwen2Model(config)
File "/home/dominik/Documents/repos/VideoLLaMA2/videollama2/model/videollama2_qwen2.py", line 42, in __init__
super(Videollama2Qwen2Model, self).__init__(config)
File "/home/dominik/Documents/repos/VideoLLaMA2/videollama2/model/videollama2_arch.py", line 34, in __init__
self.vision_tower = build_vision_tower(config)
File "/home/dominik/Documents/repos/VideoLLaMA2/videollama2/model/encoder.py", line 160, in build_vision_tower
vision_tower = SiglipVisionTower(vision_tower, args=vision_tower_cfg, **kwargs)
File "/home/dominik/Documents/repos/VideoLLaMA2/videollama2/model/encoder.py", line 99, in __init__
self.vision_tower = SiglipVisionModel(config=config)
File "/home/dominik/miniconda3/envs/vllama/lib/python3.9/site-packages/transformers/models/siglip/modeling_siglip.py", line 915, in __init__
super().__init__(config)
File "/home/dominik/miniconda3/envs/vllama/lib/python3.9/site-packages/transformers/modeling_utils.py", line 1307, in __init__
config = self._autoset_attn_implementation(
File "/home/dominik/miniconda3/envs/vllama/lib/python3.9/site-packages/transformers/modeling_utils.py", line 1454, in _autoset_attn_implementation
cls._check_and_enable_flash_attn_2(
File "/home/dominik/miniconda3/envs/vllama/lib/python3.9/site-packages/transformers/modeling_utils.py", line 1535, in _check_and_enable_flash_attn_2
raise ValueError(
ValueError: SiglipVisionModel does not support Flash Attention 2.0 yet. Please request to add support where the model is hosted, on its model hub page: https://huggingface.co//discussions/new or in the Transformers GitHub repo: https://github.com/huggingface/transformers/issues/new
```

Tried with:
- DAMO-NLP-SG/VideoLLaMA2.1-7B-16F
- DAMO-NLP-SG/VideoLLaMA2.1-7B-16F-Base
- DAMO-NLP-SG/VideoLLaMA2-7B-Base

What is the issue, why is flash attention 2.0 not supported despite being the default `_attn_implementation`

Start with own_scripts/inference.py and gradio_web_server_adhoc.py, then trace model_init in videollama2/__init__.py through load_pretrained_model in videollama2/model/__init__.py. Inspect videollama2/model/encoder.py and the SiglipVisionModel initialization to determine how _attn_implementation reaches the vision tower. Reproduce with one listed model; done when the cause and supported resolution are confirmed.

Written by the indexing model from the issue text.