[AutoDeploy]: improved failure message for unsupported quantization formats from HF

System Info

Affected Models (1):

• Qwen/Qwen2.5-7B-Instruct-AWQ

Platforms: dgxh100_coreweave
Clusters: cw-dfw-cs-001
Total Failures: 1 jobs across 1 pipeline(s)
Pipeline IDs: 40944692

Who can help?

Information

• The official example scripts
• My own modified scripts

Tasks

• An officially supported task in the examples folder
• My own task or dataset

Reproduction

Root Cause Analysis

A CUDA kernel error occurred during sampling_from_logits operation, likely due to a device-side assertion failure in the CUDA stream synchronization. This caused the sampling process to fail with an AssertionError, indicating a GPU-related runtime issue.

Steps to Reproduce

1. Run the following command:

python3 /opt/tensorrt-llm/examples/auto_deploy/build_and_run_ad.py --model Qwen/Qwen2.5-7B-Instruct-AWQ --args.yaml-extra /opt/tensorrt-llm/examples/auto_deploy/model_registry/configs/dashboard_default.yaml --args.yaml-extra /opt/tensorrt-llm/examples/auto_deploy/model_registry/configs/world_size_2.yaml --args.yaml-extra /opt/tensorrt-llm/examples/auto_deploy/model_registry/configs/compile_backend_torch_cudagraph.yaml --benchmark.enabled false --benchmark.results-path /jet/logs/basic/auto-deploy-model-coverage_m-qwen-qwen2-5-7b-instruct-awq_ws-2/extra.json --benchmark.store-results true

2. Observe failure with the error pattern shown below

Error Pattern

0: 
Processed requests:   0%|          | 0/2 [00:00<?, ?it/s]/opt/pytorch/pytorch/aten/src/ATen/native/cuda/TensorCompare.cu:112: _assert_async_cuda_kernel: block: [0,0,0], thread: [0,0,0] Assertion `` failed.

Expected behavior

The models listed above should complete inference successfully without errors.

Actual behavior

A CUDA kernel error occurred during sampling_from_logits operation, likely due to a device-side assertion failure in the CUDA stream synchronization. This caused the sampling process to fail with an AssertionError, indicating a GPU-related runtime issue.

The failure affects 1 job(s) with the same root cause pattern.

Additional notes

---

This bug ticket was auto-generated by the failure analyzer. Please review and edit before submitting.