Runtime Error

Compile and deploy successful, but runtime error:
[ComfyUI-Manager] All startup tasks have been completed.
[01:58:53] [ 51.12s] [ModelLoader] Cleaned 1575 state dict keys
[01:58:53] [ 51.12s] [ModelLoader] Applying quantization-aware replacements (quant_linear=True, fast_norm=True)...
[01:58:53] [ 51.14s] [ModelLoader] Loading weights into model...
[01:58:53] [ 51.16s] [ModelLoader] Model loaded to CPU
[01:58:53] [ 51.17s] [ModelLoader] Model wrapped with ComfyUI-native async offloading
[01:58:53] [ 51.17s] [ModelLoader] ✓ Successfully loaded model
[01:58:53] [ 51.17s] [ModelLoader] Model type: Wan2.2-A14B
[01:58:53] [ 51.17s] [ModelLoader] Attention: sla
[01:58:53] [ 51.17s] [ModelLoader] Quantized: True
[01:58:53] [ 30.91s] ✓ Model loaded in 30.91s: TurboWan2.2-I2V-A14B-high-720P-quant.pth
[01:58:53] [ 34.57s] [I2V-Inference] VRAM after high noise model loaded: 0.01GB allocated, 0.09GB reserved, 11.54GB free
[01:58:53] [ 34.57s] [I2V-Inference] Sampling with high noise model (steps 0-3)...
Sampling steps 0-3: 0%| | 0/3 [00:00<?, ?it/s]Requested to load WanModel
loaded completely; 10062.65 MB usable, 445.90 MB loaded, full load: False
terminate called after throwing an instance of 'c10::AcceleratorError'
what(): CUDA error: an illegal memory access was encountered
CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.
For debugging consider passing CUDA_LAUNCH_BLOCKING=1
Compile with TORCH_USE_CUDA_DSA to enable device-side assertions.

Exception raised from c10_cuda_check_implementation at /pytorch/c10/cuda/CUDAException.cpp:42 (most recent call first):
frame #0: c10::Error::Error(c10::SourceLocation, std::__cxx11::basic_string<char, std::char_traits, std::allocator >) + 0x80 (0x7fbb17ad9eb0 in /home/sicai/miniconda3/envs/turbodiffusion/lib/python3.12/site-packages/torch/lib/libc10.so)
frame #1: + 0x111c7 (0x7fbb17b6c1c7 in /home/sicai/miniconda3/envs/turbodiffusion/lib/python3.12/site-packages/torch/lib/libc10_cuda.so)
frame #2: + 0x560b9 (0x7fbb17bb10b9 in /home/sicai/miniconda3/envs/turbodiffusion/lib/python3.12/site-packages/torch/lib/libc10_cuda.so)
frame #3: + 0x56b64 (0x7fbb17bb1b64 in /home/sicai/miniconda3/envs/turbodiffusion/lib/python3.12/site-packages/torch/lib/libc10_cuda.so)
frame #4: + 0x434a5f (0x7fbb0a434a5f in /home/sicai/miniconda3/envs/turbodiffusion/lib/python3.12/site-packages/torch/lib/libtorch_python.so)
frame #5: c10::TensorImpl::~TensorImpl() + 0x9 (0x7fbb17ab7179 in /home/sicai/miniconda3/envs/turbodiffusion/lib/python3.12/site-packages/torch/lib/libc10.so)
frame #6: + 0xf01fda (0x7fbaf6901fda in /home/sicai/miniconda3/envs/turbodiffusion/lib/python3.12/site-packages/torch/lib/libtorch_cpu.so)
frame #7: + 0x2aed68c (0x7fbaf84ed68c in /home/sicai/miniconda3/envs/turbodiffusion/lib/python3.12/site-packages/torch/lib/libtorch_cpu.so)
frame #8: at::_ops::_to_copy::redispatch(c10::DispatchKeySet, at::Tensor const&, std::optionalc10::ScalarType, std::optionalc10::Layout, std::optionalc10::Device, std::optional, bool, std::optionalc10::MemoryFormat) + 0xf2 (0x7fbaf7bc34d2 in /home/sicai/miniconda3/envs/turbodiffusion/lib/python3.12/site-packages/torch/lib/libtorch_cpu.so)
frame #9: + 0x28ba68b (0x7fbaf82ba68b in /home/sicai/miniconda3/envs/turbodiffusion/lib/python3.12/site-packages/torch/lib/libtorch_cpu.so)
frame #10: at::_ops::_to_copy::call(at::Tensor const&, std::optionalc10::ScalarType, std::optionalc10::Layout, std::optionalc10::Device, std::optional, bool, std::optionalc10::MemoryFormat) + 0x23d (0x7fbaf7c64f7d in /home/sicai/miniconda3/envs/turbodiffusion/lib/python3.12/site-packages/torch/lib/libtorch_cpu.so)
frame #11: at::native::to(at::Tensor const&, c10::Device, c10::ScalarType, bool, bool, std::optionalc10::MemoryFormat) + 0x9b (0x7fbaf7710b4b in /home/sicai/miniconda3/envs/turbodiffusion/lib/python3.12/site-packages/torch/lib/libtorch_cpu.so)
frame #12: + 0x2be8818 (0x7fbaf85e8818 in /home/sicai/miniconda3/envs/turbodiffusion/lib/python3.12/site-packages/torch/lib/libtorch_cpu.so)
frame #13: at::_ops::to_device::call(at::Tensor const&, c10::Device, c10::ScalarType, bool, bool, std::optionalc10::MemoryFormat) + 0x1c9 (0x7fbaf7dfc309 in /home/sicai/miniconda3/envs/turbodiffusion/lib/python3.12/site-packages/torch/lib/libtorch_cpu.so)
frame #14: + 0x424d55 (0x7fbb0a424d55 in /home/sicai/miniconda3/envs/turbodiffusion/lib/python3.12/site-packages/torch/lib/libtorch_python.so)
frame #15: /home/sicai/miniconda3/envs/turbodiffusion/bin/python3.12() [0x546399]
frame #16: PyObject_Vectorcall + 0x51 (0x538e71 in /home/sicai/miniconda3/envs/turbodiffusion/bin/python3.12)
frame #17: _PyEval_EvalFrameDefault + 0x6d0 (0x521440 in /home/sicai/miniconda3/envs/turbodiffusion/bin/python3.12)
frame #18: /home/sicai/miniconda3/envs/turbodiffusion/bin/python3.12() [0x56c3a7]
frame #19: _PyEval_EvalFrameDefault + 0x50c3 (0x525e33 in /home/sicai/miniconda3/envs/turbodiffusion/bin/python3.12)
frame #20: /home/sicai/miniconda3/envs/turbodiffusion/bin/python3.12() [0x56c3a7]
frame #21: _PyEval_EvalFrameDefault + 0x50c3 (0x525e33 in /home/sicai/miniconda3/envs/turbodiffusion/bin/python3.12)
frame #22: _PyObject_FastCallDictTstate + 0x1e7 (0x519a47 in /home/sicai/miniconda3/envs/turbodiffusion/bin/python3.12)
frame #23: _PyObject_Call_Prepend + 0x66 (0x5529c6 in /home/sicai/miniconda3/envs/turbodiffusion/bin/python3.12)
frame #24: /home/sicai/miniconda3/envs/turbodiffusion/bin/python3.12() [0x627356]
frame #25: _PyObject_MakeTpCall + 0x2fc (0x5172ac in /home/sicai/miniconda3/envs/turbodiffusion/bin/python3.12)
frame #26: _PyEval_EvalFrameDefault + 0x6d0 (0x521440 in /home/sicai/miniconda3/envs/turbodiffusion/bin/python3.12)
frame #27: /home/sicai/miniconda3/envs/turbodiffusion/bin/python3.12() [0x56c3a7]
frame #28: _PyEval_EvalFrameDefault + 0x50c3 (0x525e33 in /home/sicai/miniconda3/envs/turbodiffusion/bin/python3.12)
frame #29: /home/sicai/miniconda3/envs/turbodiffusion/bin/python3.12() [0x56c3a7]
frame #30: _PyEval_EvalFrameDefault + 0x50c3 (0x525e33 in /home/sicai/miniconda3/envs/turbodiffusion/bin/python3.12)
frame #31: _PyObject_FastCallDictTstate + 0x1e7 (0x519a47 in /home/sicai/miniconda3/envs/turbodiffusion/bin/python3.12)
frame #32: _PyObject_Call_Prepend + 0x66 (0x5529c6 in /home/sicai/miniconda3/envs/turbodiffusion/bin/python3.12)
frame #33: /home/sicai/miniconda3/envs/turbodiffusion/bin/python3.12() [0x627356]
frame #34: _PyObject_MakeTpCall + 0x2fc (0x5172ac in /home/sicai/miniconda3/envs/turbodiffusion/bin/python3.12)
frame #35: _PyEval_EvalFrameDefault + 0x6d0 (0x521440 in /home/sicai/miniconda3/envs/turbodiffusion/bin/python3.12)
frame #36: /home/sicai/miniconda3/envs/turbodiffusion/bin/python3.12() [0x56c87d]
frame #37: /home/sicai/miniconda3/envs/turbodiffusion/bin/python3.12() [0x56c40d]
frame #38: _PyObject_Call + 0x122 (0x555672 in /home/sicai/miniconda3/envs/turbodiffusion/bin/python3.12)
frame #39: _PyEval_EvalFrameDefault + 0x50c3 (0x525e33 in /home/sicai/miniconda3/envs/turbodiffusion/bin/python3.12)
frame #40: /home/sicai/miniconda3/envs/turbodiffusion/bin/python3.12() [0x56c87d]
frame #41: /home/sicai/miniconda3/envs/turbodiffusion/bin/python3.12() [0x56c40d]
frame #42: _PyObject_Call + 0x122 (0x555672 in /home/sicai/miniconda3/envs/turbodiffusion/bin/python3.12)
frame #43: _PyEval_EvalFrameDefault + 0x50c3 (0x525e33 in /home/sicai/miniconda3/envs/turbodiffusion/bin/python3.12)
frame #44: _PyObject_FastCallDictTstate + 0x285 (0x519ae5 in /home/sicai/miniconda3/envs/turbodiffusion/bin/python3.12)
frame #45: _PyObject_Call_Prepend + 0x66 (0x5529c6 in /home/sicai/miniconda3/envs/turbodiffusion/bin/python3.12)
frame #46: /home/sicai/miniconda3/envs/turbodiffusion/bin/python3.12() [0x627356]
frame #47: _PyObject_Call + 0xb5 (0x555605 in /home/sicai/miniconda3/envs/turbodiffusion/bin/python3.12)
frame #48: _PyEval_EvalFrameDefault + 0x50c3 (0x525e33 in /home/sicai/miniconda3/envs/turbodiffusion/bin/python3.12)
frame #49: _PyObject_Call + 0x122 (0x555672 in /home/sicai/miniconda3/envs/turbodiffusion/bin/python3.12)
frame #50: /home/sicai/miniconda3/envs/turbodiffusion/bin/python3.12() [0x62ba4f]
frame #51: _PyObject_MakeTpCall + 0x2fc (0x5172ac in /home/sicai/miniconda3/envs/turbodiffusion/bin/python3.12)
frame #52: _PyObject_Call + 0x122 (0x555672 in /home/sicai/miniconda3/envs/turbodiffusion/bin/python3.12)
frame #53: _PyEval_EvalFrameDefault + 0x50c3 (0x525e33 in /home/sicai/miniconda3/envs/turbodiffusion/bin/python3.12)
frame #54: /home/sicai/miniconda3/envs/turbodiffusion/bin/python3.12() [0x56c87d]
frame #55: /home/sicai/miniconda3/envs/turbodiffusion/bin/python3.12() [0x56c40d]
frame #56: _PyObject_Call + 0x122 (0x555672 in /home/sicai/miniconda3/envs/turbodiffusion/bin/python3.12)
frame #57: _PyEval_EvalFrameDefault + 0x50c3 (0x525e33 in /home/sicai/miniconda3/envs/turbodiffusion/bin/python3.12)
frame #58: /home/sicai/miniconda3/envs/turbodiffusion/bin/python3.12() [0x56c87d]
frame #59: /home/sicai/miniconda3/envs/turbodiffusion/bin/python3.12() [0x56c40d]
frame #60: _PyObject_Call + 0x122 (0x555672 in /home/sicai/miniconda3/envs/turbodiffusion/bin/python3.12)
frame #61: _PyEval_EvalFrameDefault + 0x50c3 (0x525e33 in /home/sicai/miniconda3/envs/turbodiffusion/bin/python3.12)
frame #62: _PyObject_FastCallDictTstate + 0x285 (0x519ae5 in /home/sicai/miniconda3/envs/turbodiffusion/bin/python3.12)

Start by reproducing the Wan2.2-A14B inference failure during high-noise sampling, using CUDA_LAUNCH_BLOCKING=1 as suggested in the traceback. Compare the model-loading and sampling steps shown in the report and narrow down where the illegal memory access occurs. Done means runtime inference completes without the CUDA error.