Vicuna-1.5 Quantized using AWQ Not Working - CUDA Illegal Memory Access

Dear Team,

I followed the instructions at https://github.com/mit-han-lab/llm-awq#usage to quantize Vicuna-13B-1.5 model and follow the instructions at https://github.com/lm-sys/FastChat/blob/55b2f8fdb0e0b80d64e043e9fc9018641bf7289f/docs/awq.md to perform inference. I am getting CUDA Illegal Memory Access Error.

Invalid response object from API: '{"object":"error","message":"**NETWORK ERROR DUE TO HIGH TRAFFIC. PLEASE REGENERATE OR REFRESH THIS PAGE.**\\n\\n(CUDA error: an illegal memory access was encountered\\nCUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.\\nFor debugging consider passing CUDA_LAUNCH_BLOCKING=1.\\nCompile with `TORCH_USE_CUDA_DSA` to enable device-side assertions.\\n)","code":50001}' (HTTP response code was 400)

Any pointers would be greatly appreciated. Thanks

Begin with the linked llm-awq Usage instructions and FastChat docs/awq.md, then reproduce Vicuna-13B-1.5 inference with the reported configuration. Use CUDA_LAUNCH_BLOCKING=1 as suggested by the error to locate the failing call; done means identifying a reproducible cause or confirmed fix, but the issue supplies no environment details or target file.