[Usage]: how to deploy Qwen2.5-VL-7B-Instruct with triton-server

System Info

• GPU: L20
• Docker image：nvcr.io/nvidia/tritonserver:25.10-trtllm-python-py3
• Driver: 575.51.03
• CUDA version: 12.9

How would you like to use TensorRT-LLM

• command: docker run --rm -it --ipc=host --ulimit memlock=-1 --ulimit stack=67108864 --gpus=all -p8001:8001 -p8002:8002 -v /home/workspace/model:/models nvcr.io/nvidia/tritonserver:25.10-trtllm-python-py3 tritonserver --model-repository=/models

• error info
  =============================
  == Triton Inference Server ==
  =============================

NVIDIA Release 25.10 (build 227073350)
Triton Server Version 2.62.0

Copyright (c) 2018-2025, NVIDIA CORPORATION & AFFILIATES. All rights reserved.

Various files include modifications (c) NVIDIA CORPORATION & AFFILIATES. All rights reserved.

GOVERNING TERMS: The software and materials are governed by the NVIDIA Software License Agreement
(found at https://www.nvidia.com/en-us/agreements/enterprise-software/nvidia-software-license-agreement/)
and the Product-Specific Terms for NVIDIA AI Products
(found at https://www.nvidia.com/en-us/agreements/enterprise-software/product-specific-terms-for-ai-products/).

I1113 13:28:55.119790 1 pinned_memory_manager.cc:277] "Pinned memory pool is created at '0x7f43a6000000' with size 268435456"
I1113 13:28:55.120555 1 cuda_memory_manager.cc:107] "CUDA memory pool is created on device 0 with size 67108864"
I1113 13:28:55.120563 1 cuda_memory_manager.cc:107] "CUDA memory pool is created on device 1 with size 67108864"
I1113 13:28:55.120569 1 cuda_memory_manager.cc:107] "CUDA memory pool is created on device 2 with size 67108864"
I1113 13:28:55.120574 1 cuda_memory_manager.cc:107] "CUDA memory pool is created on device 3 with size 67108864"
W1113 13:28:55.817451 1 model_lifecycle.cc:112] "ignore version directory '.git' which fails to convert to integral number"
E1113 13:28:55.817501 1 model_repository_manager.cc:724] "failed to load model 'Qwen2.5-VL-7B-Instruct': at least one version must be available under the version policy of model 'Qwen2.5-VL-7B-Instruct'"
W1113 13:28:55.819409 1 model_lifecycle.cc:112] "ignore version directory '.git' which fails to convert to integral number"
E1113 13:28:55.819418 1 model_repository_manager.cc:724] "failed to load model 'Qwen2.5-VL-7B-Instruct-AWQ': at least one version must be available under the version policy of model 'Qwen2.5-VL-7B-Instruct-AWQ'"
I1113 13:28:55.819457 1 server.cc:611]
+------------------+------+
| Repository Agent | Path |
+------------------+------+
+------------------+------+

I1113 13:28:55.819504 1 server.cc:638]
+---------+------+--------+
| Backend | Path | Config |
+---------+------+--------+
+---------+------+--------+

I1113 13:28:55.819526 1 server.cc:681]
+----------------------------+---------+----------------------------------------+
| Model | Version | Status |
+----------------------------+---------+----------------------------------------+
| Qwen2.5-VL-7B-Instruct | - | Not loaded: No model version was found |
| Qwen2.5-VL-7B-Instruct-AWQ | - | Not loaded: No model version was found |
+----------------------------+---------+----------------------------------------+

I1113 13:28:55.863542 1 metrics.cc:889] "Collecting metrics for GPU 0: NVIDIA L20"
I1113 13:28:55.863567 1 metrics.cc:889] "Collecting metrics for GPU 1: NVIDIA L20"
I1113 13:28:55.863575 1 metrics.cc:889] "Collecting metrics for GPU 2: NVIDIA L20"
I1113 13:28:55.863581 1 metrics.cc:889] "Collecting metrics for GPU 3: NVIDIA L20"
I1113 13:28:55.874394 1 metrics.cc:782] "Collecting CPU metrics"
I1113 13:28:55.874787 1 tritonserver.cc:2598]
+----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Option | Value |
+----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| server_id | triton |
| server_version | 2.62.0 |
| server_extensions | classification sequence model_repository model_repository(unload_dependents) schedule_policy model_configuration system_shared_memory cuda_shared_memory binary_tens |
| | or_data parameters statistics trace logging |
| model_repository_path[0] | /models |
| model_control_mode | MODE_NONE |
| strict_model_config | 0 |
| model_config_name | |
| rate_limit | OFF |
| pinned_memory_pool_byte_size | 268435456 |
| cuda_memory_pool_byte_size{0} | 67108864 |
| cuda_memory_pool_byte_size{1} | 67108864 |
| cuda_memory_pool_byte_size{2} | 67108864 |
| cuda_memory_pool_byte_size{3} | 67108864 |
| min_supported_compute_capability | 6.0 |
| strict_readiness | 1 |
| exit_timeout | 30 |
| cache_enabled | 0 |
+----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------+

I1113 13:28:55.874907 1 server.cc:312] "Waiting for in-flight requests to complete."
I1113 13:28:55.874913 1 server.cc:328] "Timeout 30: Found 0 model versions that have in-flight inferences"
I1113 13:28:55.874965 1 server.cc:343] "All models are stopped, unloading models"
I1113 13:28:55.874971 1 server.cc:352] "Timeout 30: Found 0 live models and 0 in-flight non-inference requests"
error: creating server: Internal - failed to load all models

Before submitting a new issue...

• Make sure you already searched for relevant issues, and checked the documentation and examples for answers to frequently asked questions.