ktransformers cannot be used in SGLang

### Reminder

- [x] I have read the above rules and searched the existing issues.

### System Info

docker image:

> lmsysorg/sglang:v0.5.5.post3

nvidia info in docker instance:

> +-----------------------------------------------------------------------------------------+
> | NVIDIA-SMI 550.90.07 Driver Version: 550.90.07 CUDA Version: 12.9 |
> |-----------------------------------------+------------------------+----------------------+
> | GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC |
> | Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. |
> | | | MIG M. |
> |=========================================+========================+======================|
> | 0 NVIDIA L40 On | 00000000:34:00.0 Off | 0 |
> | N/A 27C P8 34W / 300W | 1MiB / 46068MiB | 0% Default |
> | | | N/A |
> +-----------------------------------------+------------------------+----------------------+
> | 1 NVIDIA L40 On | 00000000:35:00.0 Off | 0 |
> | N/A 29C P8 34W / 300W | 1MiB / 46068MiB | 0% Default |
> | | | N/A |
> +-----------------------------------------+------------------------+----------------------+
> | 2 NVIDIA L40 On | 00000000:37:00.0 Off | 0 |
> | N/A 28C P8 33W / 300W | 1MiB / 46068MiB | 0% Default |
> | | | N/A |
> +-----------------------------------------+------------------------+----------------------+
> | 3 NVIDIA L40 On | 00000000:9C:00.0 Off | 0 |
> | N/A 28C P8 33W / 300W | 1MiB / 46068MiB | 0% Default |
> | | | N/A |
> +-----------------------------------------+------------------------+----------------------+
> | 4 NVIDIA L40 On | 00000000:9D:00.0 Off | 0 |
> | N/A 30C P8 35W / 300W | 1MiB / 46068MiB | 0% Default |
> | | | N/A |
> +-----------------------------------------+------------------------+----------------------+
> | 5 NVIDIA L40 On | 00000000:9E:00.0 Off | 0 |
> | N/A 26C P8 32W / 300W | 1MiB / 46068MiB | 0% Default |
> | | | N/A |
> +-----------------------------------------+------------------------+----------------------+

ktransformers version:

master code for commit: 637c49c83fe13aa4f4b1d227e464ae2293ec900f
I just use ktransformers/kt-kernel/install.sh to build kt-kernel

model info:

Qwen3-30B-A3B-Thinking-2507

gguf model info:

Qwen3-30B-A3B-Thinking-2507-F16.gguf

cpu info:

cpu just support avx ;128 core; 2 numa node

memory info: 2015G

### Reproduction

```text
python -m sglang.launch_server --model-path /model/qwen3/Qwen3-30B-A3B-Thinking-2507 --host 0.0.0.0 --port 30000 --kt-method LLAMAFILE --kt-weight-path /model/qwen3/Qwen3-30B-A3B-Thinking-2507--gguf/ --kt-cpuinfer 32 --kt-threadpool-count 1 --kt-num-gpu-experts 0 --tp 1 --log-level debug --trust-remote-code --enable-cache-report --kt-max-deferred-experts-per-token 0
```

### Others

I want to run qwen3-30b-a3b model by gpu and cpu; ktransformers model weight is qwen3-30b-a3b-f16; the sglang program crash and stack is below; if I do not use ktransformers, everything is ok;

ang when sglang program hang, I find many progress is 'defunct';such as:

> root 9372 1 0 08:50 pts/0 00:00:00 [python]
> root 9389 9182 08:50 pts/0 00:00:00 /usr/bin/python /usr/local/lib/python3.12/dist-packages/torch/_inductor/compile_worker/__main__.py --pickler=torch._inductor.compile_worker.subproc_pool.....................

sglang error info:

> Capturing batches (bs=32 avail_mem=3.90 GB): 0%| | 0/8 [00:00 [2025-12-01 09:18:36] Lock 140112113636592 acquired on /root/.cache/flashinfer/0.5.2/89/cached_ops/tmp/batch_prefill_with_kv_cache_dtype_q_bf16_dtype_kv_bf16_dtype_o_bf16_dtype_idx_i32_head_dim_qk_128_head_dim_vo_128_posenc_0_use_swa_False_use_logits_cap_False_f16qk_False.lock
> [2025-12-01 09:18:36] Attempting to release lock 140112113636592 on /root/.cache/flashinfer/0.5.2/89/cached_ops/tmp/batch_prefill_with_kv_cache_dtype_q_bf16_dtype_kv_bf16_dtype_o_bf16_dtype_idx_i32_head_dim_qk_128_head_dim_vo_128_posenc_0_use_swa_False_use_logits_cap_False_f16qk_False.lock
> [2025-12-01 09:18:36] Lock 140112113636592 released on /root/.cache/flashinfer/0.5.2/89/cached_ops/tmp/batch_prefill_with_kv_cache_dtype_q_bf16_dtype_kv_bf16_dtype_o_bf16_dtype_idx_i32_head_dim_qk_128_head_dim_vo_128_posenc_0_use_swa_False_use_logits_cap_False_f16qk_False.lock
> Fatal Python error: Fatal Python error: Fatal Python error: Fatal Python error: Fatal Python error: Fatal Python error: Fatal Python error: Fatal Python error: Fatal Python error: Fatal Python error: Fatal Python error: Fatal Python error: Fatal Python error: Fatal Python error: Fatal Python error: Fatal Python error: Fatal Python error: Fatal Python error: Fatal Python error: Fatal Python error: Fatal Python error: Fatal Python error: Fatal Python error: Illegal instructionFatal Python error: Fatal Python error: Fatal Python error: Fatal Python error: Illegal instructionFatal Python error: Fatal Python error: Fatal Python error: Illegal instructionFatal Python error: Fatal Python error: Illegal instructionIllegal instructionIllegal instructionIllegal instructionIllegal instructionIllegal instructionIllegal instructionIllegal instructionIllegal instructionIllegal instructionIllegal instructionIllegal instructionIllegal instructionIllegal instructionIllegal instructionIllegal instructionIllegal instructionIllegal instructionIllegal instructionIllegal instruction
>
> Illegal instructionIllegal instructionIllegal instructionIllegal instruction
>
> Illegal instructionIllegal instructionIllegal instruction
>
> Illegal instructionIllegal instruction
>
> Thread 0x
>
> 00007f394bfff640 (most recent call first):
> File "/usr/lib/python3.12/threading.py", line 359 in wait
> File "/usr/lib/python3.12/threading.py", line 655 in wait
> File "/usr/local/lib/python3.12/dist-packages/tqdm/_monitor.py", line 60 in run
> File "/usr/lib/python3.12/threading.py", line 1075 in _bootstrap_inner
> File "/usr/lib/python3.12/threading.py", line 1032 in _bootstrap
>
> Extension modules:
> numpy._core._multiarray_umathThread 0x00007f5e3ffff640 (most recent call first):
> File "/usr/lib/python3.12/threading.py", line 359 in , waitnumpy.linalg._umath_linalg
> File "/usr/lib/python3.12/threading.py", line 655 in wait
> File "/usr/local/lib/python3.12/dist-packages/tqdm/_monitor.py", line 60 in run
> File "/usr/lib/python3.12/threading.py", line 1075 in _bootstrap_inner
> File "/usr/lib/python3.12/threading.py", line 1032 in _bootstrap
>
> Thread 0x00007f7e003b0640 (most recent call first):
>
> Extension modules: numpy._core._multiarray_umath File "/usr/local/lib/python3.12/dist-packages/torch/_inductor/compile_worker/subproc_pool.py", , line numpy.linalg._umath_linalg61 in _recv_msg
> File "/usr/local/lib/python3.12/dist-packages/torch/_inductor/compile_worker/subproc_pool.py", line 195 in _read_thread
> File "/usr/lib/python3.12/threading.py", line 1012, in pybase64._pybase64run
> File "/usr/lib/python3.12/threading.py", line 1075 in _bootstrap_inner
> File "/usr/lib/python3.12/threading.py", line 1032 in _bootstrap
>
> Thread 0x00007f8267322480 (most recent call first):
> File ", /usr/lib/python3.12/tokenize.py,
> Extension modules: charset_normalizer.md"pybase64._pybase64numpy._core._multiarray_umath, line
> Extension modules: 459
> Extension modules: numpy._core._multiarray_umath
> Extension modules:
> Extension modules: in ,
> Extension modules: ,
> Extension modules:
> Extension modules: ,
> Extension modules:
> Extension modules:
> Extension modules:
> Extension modules: numpy._core._multiarray_umath
> Extension modules:
> Extension modules:
> Extension modules:
> Extension modules: numpy._core._multiarray_umath
> Extension modules: numpy._core._multiarray_umathnumpy._core._multiarray_umath
> Extension modules:
> Extension modules: openrequests.packages.charset_normalizer.md, , numpy._core._multiarray_umathcharset_normalizer.mdnumpy._core._multiarray_umath
> Extension modules:
> Extension modules: numpy._core._multiarray_umath
> Extension modules: numpy.linalg._umath_linalgnumpy._core._multiarray_umath
> Extension modules:
> Extension modules: numpy._core._multiarray_umathnumpy._core._multiarray_umathnumpy._core._multiarray_umath
> Extension modules: numpy._core._multiarray_umath, numpy._core._multiarray_umathnumpy._core._multiarray_umath
> Extension modules: numpy._core._multiarray_umath, numpy._core._multiarray_umath, numpy._core._multiarray_umath
> numpy.linalg._umath_linalgnumpy.linalg._umath_linalg, , numpy._core._multiarray_umathnumpy._core._multiarray_umath, , numpy._core._multiarray_umath, , numpy._core._multiarray_umathnumpy._core._multiarray_umath, numpy._core._multiarray_umath,
> Extension modules:
> Extension modules:
> Extension modules: numpy.linalg._umath_linalg, numpy._core._multiarray_umath, numpy.linalg._umath_linalg, , numpy.linalg._umath_linalg File , , numpy.linalg._umath_linalg, requests.packages.chardet.mdnumpy.linalg._umath_linalgrequests.packages.charset_normalizer.mdnumpy.linalg._umath_linalgnumpy.linalg._umath_linalg, numpy.linalg._umath_linalg, numpy.linalg._umath_linalg, , numpy.linalg._umath_linalg, numpy.linalg._umath_linalgnumpy._core._multiarray_umath, numpy._core._multiarray_umathnumpy._core._multiarray_umath, numpy.linalg._umath_linalgnumpy.linalg._umath_linalg, numpy.linalg._umath_linalgnumpy.linalg._umath_linalg, "numpy.linalg._umath_linalgnumpy.linalg._umath_linalgnumpy.linalg._umath_linalg, , , numpy.linalg._umath_linalgnumpy.linalg._umath_linalg, pybase64._pybase64numpy.linalg._umath_linalg, numpy.linalg._umath_linalg, , numpy.linalg._umath_linalgpybase64._pybase64, , , , pybase64._pybase64, pybase64._pybase64requests.packages.chardet.md, , , , pybase64._pybase64pybase64._pybase64numpy.linalg._umath_linalgpybase64._pybase64pybase64._pybase64, , numpy.linalg._umath_linalgnumpy.linalg._umath_linalg, "pybase64._pybase64, , pybase64._pybase64, charset_normalizer.mdpybase64._pybase64, pybase64._pybase64pybase64._pybase64, multidict._multidict, charset_normalizer.md, pybase64._pybase64, , , , charset_normalizer.md, , line charset_normalizer.md, pybase64._pybase64, , , , pybase64._pybase64, , pybase64._pybase64pybase64._pybase64, , , , charset_normalizer.md, , , charset_normalizer.md, charset_normalizer.md, pybase64._pybase64pybase64._pybase64, pybase64._pybase64charset_normalizer.md1pybase64._pybase64, pybase64._pybase64, charset_normalizer.mdpybase64._pybase64charset_normalizer.md, pybase64._pybase64requests.packages.charset_normalizer.mdmultidict._multidictcharset_normalizer.md, , , charset_normalizer.mdcharset_normalizer.mdyarl._quoting_cpybase64._pybase64pybase64._pybase64charset_normalizer.md, , pybase64._pybase64, , , requests.packages.charset_normalizer.md, , in , , requests.packages.charset_normalizer.md, charset_normalizer.md, , requests.packages.charset_normalizer.md, , , charset_normalizer.mdrequests.packages.chardet.mdcharset_normalizer.md, , , , , , , requests.packages.charset_normalizer.md, , pybase64._pybase64charset_normalizer.mdcharset_normalizer.mdrequests.packages.charset_normalizer.md, requests.packages.charset_normalizer.md, requests.packages.charset_normalizer.md, , , , charset_normalizer.mdcharset_normalizer.mdrequests.packages.charset_normalizer.md, charset_normalizer.mdcharset_normalizer.mdrequests.packages.charset_normalizer.mdrequests.packages.charset_normalizer.md, charset_normalizer.mdrequests.packages.chardet.mdyarl._quoting_ccharset_normalizer.mdrequests.packages.charset_normalizer.mdcharset_normalizer.md, , requests.packages.charset_normalizer.mdrequests.packages.charset_normalizer.mdpropcache._helpers_c, requests.packages.charset_normalizer.md, requests.packages.chardet.md, , , charset_normalizer.md
> requests.packages.chardet.mdcharset_normalizer.mdcharset_normalizer.md, , requests.packages.chardet.md, , , requests.packages.charset_normalizer.md, , , , propcache._helpers_crequests.packages.charset_normalizer.md, , , , , , requests.packages.charset_normalizer.md, requests.packages.chardet.md, requests.packages.charset_normalizer.mdrequests.packages.chardet.mdrequests.packages.charset_normalizer.md, requests.packages.chardet.md File multidict._multidict", requests.packages.chardet.md, requests.packages.chardet.md, requests.packages.charset_normalizer.mdrequests.packages.charset_normalizer.mdrequests.packages.chardet.mdrequests.packages.chardet.mdrequests.packages.charset_normalizer.mdrequests.packages.charset_normalizer.md, requests.packages.charset_normalizer.mdrequests.packages.chardet.md, requests.packages.charset_normalizer.mdaiohttp._http_writer, , requests.packages.chardet.md, requests.packages.chardet.mdaiohttp._http_writer, requests.packages.chardet.md, /usr/lib/python3.12/linecache.py, requests.packages.charset_normalizer.mdrequests.packages.charset_normalizer.mdrequests.packages.charset_normalizer.md, , , , requests.packages.chardet.mdrequests.packages.chardet.md, , multidict._multidict, , requests.packages.chardet.md, , multidict._multidict, , , requests.packages.chardet.md, , multidict._multidict"requests.packages.chardet.md, yarl._quoting_c, multidict._multidictrequests.packages.chardet.md, , requests.packages.chardet.md, , requests.packages.chardet.mdmultidict._multidictrequests.packages.chardet.mdmultidict._multidictaiohttp._http_parserrequests.packages.chardet.mdyarl._quoting_c, multidict._multidictmultidict._multidictmultidict._multidictaiohttp._http_parser, , yarl._quoting_c, line multidict._multidictrequests.packages.chardet.md, , , , propcache._helpers_crequests.packages.chardet.mdmultidict._multidictrequests.packages.chardet.mdmultidict._multidict, , , , multidict._multidict, , , yarl._quoting_c, , multidict._multidict, , , , , , 141yarl._quoting_cmultidict._multidict, propcache._helpers_c, , yarl._quoting_cyarl._quoting_cmultidict._multidictaiohttp._websocket.mask, , , multidict._multidict, , multidict._multidict, scipy.spatial._hausdorffscipy.spatial._voronoiscipy.spatial._ckdtreescipy.spatial._ckdtreescipy.spatial._distance_wrapscipy.spatial._qhullscipy.optimize._bglu_densescipy.spatial._ckdtreescipy.spatial._voronoi, , scipy.optimize._lsap, , scipy.spatial._voronoi, scipy.spatial._ckdtree, , , scipy.spatial._voronoi, , , , scipy.spatial._voronoiscipy.spatial._voronoi, scipy.spatial._ckdtreescipy.optimize._moduleTNC, , scipy.spatial._hausdorffscipy.spatial._voronoi, , , , , , scipy.spatial._hausdorff, scipy.spatial._qhull, , , scipy.optimize._bglu_densescipy.spatial._ckdtreescipy.spatial._hausdorffscipy.spatial._hausdorffscipy.spatial._voronoi, scipy.spatial._qhull, scipy.spatial._hausdorffscipy.spatial._ckdtree, scipy.spatial._ckdtreescipy.spatial._ckdtree, , , scipy.spatial._voronoi, scipy.spatial._hausdorffscipy.spatial._hausdorff, scipy.spatial._voronoi, scipy.spatial._qhullscipy.spatial._qhullscipy.spatial._distance_wrap, scipy.spatial._hausdorffscipy.optimize._lsapscipy.spatial._hausdorffscipy.spatial.transform._rotationscipy.spatial._qhull, , scipy.spatial._distance_wrap, , scipy.spatial._distance_wrapscipy.spatial._qhull, , , scipy.spatial._hausdorff, , scipy.spatial._hausdorff, scipy.spatial._hausdorffscipy.spatial._qhull, scipy.optimize._slsqplibscipy.spatial._distance_wrap, , scipy.spatial._hausdorff, scipy.spatial._hausdorff, , , , scipy.spatial.transform._rotation, , scipy.spatial._voronoiscipy.spatial._qhull, , scipy.optimize._lsapscipy.spatial._hausdorff, , , scipy.spatial._distance_wrapscipy.spatial._voronoiscipy.spatial._distance_wrapscipy.spatial._qhull, scipy.spatial._qhull, scipy.spatial._qhull, , scipy.spatial._distance_wrap, scipy.spatial._distance_wrap, scipy.spatial._hausdorffscipy.spatial._voronoiscipy.spatial._voronoi, scipy.spatial._distance_wrapscipy.spatial._ckdtree, scipy.spatial._distance_wrap, scipy.spatial.transform._rigid_transform, scipy.spatial._voronoi, , scipy.spatial.transform._rotationscipy.spatial.transform._rotationscipy.spatial._voronoi, , , , scipy.spatial._distance_wrap, , scipy.spatial.transform._rotationscipy.spatial._distance_wrap, scipy.spatial._distance_wrapscipy.spatial._voronoi, scipy.optimize._minpack, , scipy.spatial.transform._rotation, , scipy.spatial._distance_wrap, , scipy.spatial._distance_wrap, , scipy.spatial.transform._rigid_transform, scipy.spatial._voronoiscipy.spatial._hausdorffscipy.spatial.transform._rotation, , , , scipy.spatial._ckdtreescipy.spatial._distance_wrap, scipy.spatial.transform._rigid_transformscipy.spatial._hausdorffscipy.spatial._voronoi, scipy.spatial.transform._rotation, scipy.spatial._voronoi, , scipy.spatial._voronoi, scipy.spatial.transform._rigid_transformscipy.spatial.transform._rotationscipy.spatial.transform._rotation, , scipy.spatial._hausdorffscipy.spatial._distance_wrapscipy.spatial.transform._rotation, scipy.spatial._hausdorff, scipy.spatial._qhull, scipy.optimize._directscipy.spatial._hausdorff, scipy.spatial.transform._rigid_transform, scipy.optimize._directscipy.spatial._hausdorff, , , scipy.spatial.transform._rotation, , scipy.spatial.transform._rotationscipy.spatial.transform._rotationscipy.spatial._hausdorff, scipy.optimize._lsq.givens_elimination, scipy.spatial.transform._rigid_transform, scipy.spatial._hausdorff, , , scipy.spatial.transform._rotation, , , scipy.spatial.transform._rotation, scipy.spatial._hausdorff, scipy.spatial._distance_wrap, scipy.spatial.transform._rigid_transform, , scipy.spatial._qhullscipy.spatial.transform._rotationscipy.optimize._direct, scipy.spatial._distance_wrap, scipy.spatial._hausdorff, scipy.spatial.transform._rigid_transform, scipy.spatial._hausdorff, , scipy.spatial.transform._rigid_transform, scipy.optimize._directscipy.spatial.transform._rigid_transformscipy.spatial._distance_wrap, scipy.spatial.transform._rotation, scipy.spatial.transform._rigid_transformsetproctitle._setproctitlescipy.spatial._distance_wrapscipy.spatial._distance_wrapscipy.spatial.transform._rigid_transform, , scipy.spatial._voronoiscipy.spatial._distance_wrapscipy.optimize._direct, , scipy.spatial.transform._rotation, scipy.spatial._distance_wrap, , scipy.spatial.transform._rigid_transformscipy.spatial.transform._rigid_transform, scipy.spatial.transform._rigid_transformscipy.spatial._distance_wrap, scipy.spatial.transform._rigid_transform, , scipy.optimize._zerosscipy.optimize._direct, , , setproctitle._setproctitle, , , , scipy.spatial._distance_wrapscipy.spatial.transform._rigid_transform, , , , scipy.spatial._hausdorff, scipy.optimize._directscipy.spatial._voronoi, scipy.spatial.transform._rotationscipy.spatial.transform._rigid_transform, scipy.spatial._distance_wrapscipy.spatial.transform._rotation, , , , setproctitle._setproctitlesetproctitle._setproctitlescipy.optimize._direct, scipy.optimize._direct, scipy.spatial._distance_wrapscipy.optimize._directscipy.spatial.transform._rotationscipy.optimize._directscipy.spatial.transform._rigid_transform, , setproctitle._setproctitlescipy.spatial.transform._rotation, scipy.spatial.transform._rotation, scipy.optimize._direct, scipy.spatial.transform._rotation, , , , , , scipy.optimize._directscipy.optimize._directscipy.optimize._directscipy.spatial.transform._rotation, , , scipy.optimize._directscipy._lib._uarray._uarray, setproctitle._setproctitlescipy.spatial.transform._rotation, , , scipy.spatial.transform._rotation, , , , scipy.spatial.transform._rigid_transform, scipy.optimize._direct, core, scipy.spatial._distance_wrapcorescipy.spatial.transform._rigid_transformscipy.spatial._hausdorffscipy.spatial.transform._rotation, scipy.optimize._directscipy.spatial.transform._rigid_transformcore, , setproctitle._setproctitlescipy.spatial.transform._rigid_transform, , core, scipy.optimize._direct, setproctitle._setproctitle, setproctitle._setproctitlesetproctitle._setproctitlesetproctitle._setproctitle, scipy.spatial.transform._rigid_transformsetproctitle._setproctitle, scipy.spatial.transform._rigid_transform, , , , scipy.spatial.transform._rigid_transform, , , core, scipy.spatial.transform._rigid_transform, , setproctitle._setproctitlescipy.special._ufuncs_cxx, , scipy.spatial.transform._rigid_transformsetproctitle._setproctitle, setproctitle._setproctitlesetproctitle._setproctitle, scipy.optimize._directcore, , , , cuda.bindings._bindings.cydriverscipy.optimize._direct, scipy.spatial.transform._rotation, scipy.spatial.transform._rigid_transformcore, , , corecuda.bindings._bindings.cydriver, , scipy.spatial._distance_wrapsetproctitle._setproctitlescipy.optimize._directcuda.bindings._bindings.cydriver, , scipy.optimize._directsetproctitle._setproctitlecuda.bindings._bindings.cydriver, , setproctitle._setproctitlescipy.optimize._directcorecore, scipy.optimize._directcore, , , , , core, , core, setproctitle._setproctitle, scipy.optimize._direct, , , scipy.optimize._direct, cuda.bindings._bindings.cydriver, scipy.optimize._directscipy.special._ellip_harm_2, , cuda.bindings._bindings.cydriver, , core, , , cuda.bindings.cydriversetproctitle._setproctitlecorescipy.spatial.transform._rigid_transform, , scipy.optimize._directcore, , core, cuda.bindings._bindings.cydrivercuda.bindings._bindings.cydriver, cuda.bindings._bindings.cydrivercuda.bindings.cydriver, scipy.spatial.transform._rotationcuda.bindings.cydrivercuda.bindings.cydriversetproctitle._setproctitle, , , cuda.bindings.cydriversetproctitle._setproctitle, , cuda.bindings._bindings.cydriver, cuda.bindings.cydrivercore, cuda.bindings._bindings.cydriver, , cuda.bindings._bindings.cydrivercore, setproctitle._setproctitlescipy.optimize._direct, setproctitle._setproctitle, cuda.bindings._bindings.cydriver, , , core, corecuda.bindings.cydriver, , , , cuda.bindings.drivercuda.bindings.driver, setproctitle._setproctitlesetproctitle._setproctitlescipy.special._special_ufuncs, , cuda.bindings.driver, setproctitle._setproctitlecuda.bindings._bindings.cydrivercuda.bindings.drivercuda.bindings._bindings.cydriver, , , cuda.bindings._bindings.cydriver, , cuda.bindings.cydrivercuda.bindings._bindings.cydriversetproctitle._setproctitlecuda.bindings.cydriver, cuda.bindings._bindings.cydriver, , cuda.bindings.cydriver, cuda.bindings._bindings.cydriver, , scipy.spatial.transform._rigid_transformcuda.bindings.driver, , cuda.bindings.cydriver, , cuda.bindings.driver, , scipy.special._gufuncs, , corecuda.bindings.cydriver, , core, cuda.bindings._bindings.cydriver, , core, corecuda.bindings.cydriver, , cuda.bindings.cydriver, setproctitle._setproctitlecore, cuda.bindings._bindings.cydrivercuda.bindings.driver, , tvm_ffi.corecore, scipy.optimize._directtvm_ffi.core, , cuda.bindings.cydrivertvm_ffi.coretvm_ffi.core, , cuda.bindings.cydrivercoretvm_ffi.core, , cuda.bindings._bindings.cydriver, , cuda.bindings.cydrivercuda.bindings.driver, , cuda.bindings.driver, cuda.bindings.cydriver, , cuda.bindings.cydriver, cuda.bindings.drivercore, cuda.bindings.drivertvm_ffi.core, scipy.special._ufuncs, cuda.bindings.driver, , cuda.bindings.driver, cuda.bindings._bindings.cydriver, , , cuda.bindings.cydrivermsgspec._corecuda.bindings._bindings.cydriver, , cuda.bindings._bindings.cydriver, cuda.bindings.driver, cuda.bindings.driver, cuda.bindings.cydrivercuda.bindings._bindings.cydriver, , , cuda.bindings.drivertvm_ffi.core, cuda.bindings._bindings.cydriver, , msgspec._corecore, , cuda.bindings.driver, , msgspec._core, , setproctitle._setproctitlecuda.bindings._bindings.cydrivermsgspec._core, , , msgspec._core, cuda.bindings.cydrivercuda.bindings.drivertvm_ffi.coretvm_ffi.core, cuda.bindings.driver, cuda.bindings.driver, , cuda.bindings._bindings.cydrivertvm_ffi.coretvm_ffi.core, scipy.special._specfuncuda.bindings._bindings.cynvrtc, msgspec._coretvm_ffi.core, msgspec._coretvm_ffi.core, cuda.bindings.cydriver, tvm_ffi.core, cuda.bindings.drivercuda.bindings.cydriver, cuda.bindings.cydriver, , , , , tvm_ffi.core, , cuda.bindings.cydriver, tvm_ffi.core, tvm_ffi.core, , , cuda.bindings.cydriver, , cuda.bindings._bindings.cydrivercuda.bindings._bindings.cynvrtc, cuda.bindings.cydriver, cuda.bindings._bindings.cynvrtc, corecuda.bindings._bindings.cynvrtc, , , cuda.bindings.driver, , cuda.bindings._bindings.cynvrtccuda.bindings._bindings.cynvrtc, cuda.bindings._bindings.cynvrtctvm_ffi.coremsgspec._coretvm_ffi.coremsgspec._coretvm_ffi.core, , cuda.bindings.cydriverscipy.special._combmsgspec._coremsgspec._core, , cuda.bindings.cynvrtcmsgspec._core, cuda.bindings.driver, , , cuda.bindings.cynvrtcmsgspec._coretvm_ffi.core, , , cuda.bindings.drivermsgspec._corecuda.bindings.driver, , , , msgspec._corecuda.bindings.driver, , , , , msgspec._coremsgspec._core, cuda.bindings.driver, , cuda.bindings.cynvrtccuda.bindings.cydriver, cuda.bindings.driver, , , cuda.bindings.nvrtccuda.bindings._bindings.cydrivercuda.bindings.cynvrtctvm_ffi.core, cuda.bindings.cynvrtc, , cuda.bindings.cynvrtccuda.bindings.cynvrtc, , cuda.bindings._bindings.cynvrtcmsgspec._coremsgspec._corecuda.bindings.driver, msgspec._corecuda.bindings._bindings.cynvrtc, , cuda.bindings._bindings.cynvrtcscipy.linalg._decomp_interpolativecuda.bindings._bindings.cynvrtccuda.bindings.nvrtc, , , , tvm_ffi.core, tvm_ffi.core, cuda.bindings._bindings.cynvrtccuda.bindings._bindings.cynvrtctvm_ffi.core, , , msgspec._core, cuda.bindings._bindings.cynvrtc, cuda.bindings.nvrtctvm_ffi.core, , , , tvm_ffi.corecuda.bindings._bindings.cynvrtc, , , , cuda.bindings.nvrtc, , , cuda.bindings._bindings.cynvrtccuda.bindings.cynvrtccuda.bindings.drivercuda.bindings._bindings.cynvrtc, tvm_ffi.corecuda.bindings.nvrtc, , , cuda.bindings.cydrivertvm_ffi.core, cuda.bindings._bindings.cyruntime_ptds, msgspec._coremsgspec._core, cuda.bindings.nvrtccuda.bindings.nvrtc, , cuda.bindings.cynvrtctvm_ffi.core, cuda.bindings._bindings.cynvrtc, cuda.bindings._bindings.cynvrtc, cuda.bindings.cynvrtc, , cuda.bindings._bindings.cynvrtc, scipy.optimize._bglu_dense, cuda.bindings.cynvrtc, , cuda.bindings._bindings.cyruntime_ptdscuda.bindings.nvrtcmsgspec._core, , , cuda.bindings.cynvrtccuda.bindings.cynvrtc, , msgspec._core, cuda.bindings.cynvrtccuda.bindings._bindings.cynvrtc, , cuda.bindings._bindings.cyruntime_ptds, cuda.bindings.nvrtc, , cuda.bindings.cynvrtc, , msgspec._corecuda.bindings.cynvrtcmsgspec._core, cuda.bindings.nvrtc, cuda.bindings._bindings.cyruntime_ptds, cuda.bindings.cynvrtc, cuda.bindings.cynvrtccuda.bindings._bindings.cyruntime_ptds, msgspec._core, cuda.bindings._bindings.cyruntime, , cuda.bindings.driver, msgspec._core, cuda.bindings._bindings.cyruntime_ptdscuda.bindings._bindings.cynvrtc, , cuda.bindings._bindings.cyruntime_ptdscuda.bindings._bindings.cynvrtc, cuda.bindings._bindings.cynvrtc, msgspec._corecuda.bindings.cynvrtccuda.bindings.cynvrtc, scipy.optimize._lsap, cuda.bindings.nvrtc, , cuda.bindings.nvrtc, , cuda.bindings._bindings.cyruntime_ptds, cuda.bindings._bindings.cyruntime, cuda.bindings.nvrtc, cuda.bindings._bindings.cynvrtccuda.bindings.cyruntimecuda.bindings.nvrtc, , cuda.bindings.nvrtccuda.bindings.cynvrtc, , cuda.bindings._bindings.cyruntime, , cuda.bindings._bindings.cyruntime_ptds, cuda.bindings.nvrtc, , cuda.bindings.nvrtc, cuda.bindings._bindings.cyruntime, cuda.bindings._bindings.cynvrtccuda.bindings._bindings.cynvrtccuda.bindings._bindings.cyruntime_ptds, , cuda.bindings.nvrtccuda.bindings.nvrtc, cuda.bindings._bindings.cyruntime, , cuda.bindings.cyruntime, cuda.bindings._bindings.cynvrtc, tvm_ffi.corecuda.bindings.cynvrtc, , cuda.bindings._bindings.cyruntime, cuda.bindings._bindings.cynvrtccuda.bindings._bindings.cyruntimecuda.bindings.cynvrtc, , cuda.bindings.nvrtccuda.bindings.cynvrtc, cuda.bindings._bindings.cynvrtcscipy.spatial._ckdtree, , , cuda.bindings._bindings.cyruntime_ptdscuda.bindings._bindings.cyruntime_ptds, cuda.bindings._bindings.cyruntime, , cuda.bindings._bindings.cyruntime_ptds, cuda.bindings.cynvrtc, , cuda.bindings._bindings.cyruntime_ptdscuda.bindings.runtime, , cuda.bindings._bindings.cyruntime_ptds, , cuda.bindings._bindings.cyruntime_ptds, cuda.bindings.nvrtc, cuda.bindings.cyruntimecuda.bindings._bindings.cyruntime, , , cuda.bindings._bindings.cyruntime_ptds, cuda.bindings.cyruntime, , cuda.bindings._bindings.cyruntime_ptdscuda.bindings.cynvrtc, cuda.bindings.cynvrtc, , cuda.bindings._bindings.cyruntime, cuda.bindings._bindings.cyruntime_ptds, cuda.bindings.cyruntimecuda.bindings._bindings.cyruntime, cuda.bindings.cynvrtccuda.bindings.runtime, cuda.bindings.nvrtccuda.bindings._bindings.cyruntime, , msgspec._corecuda.bindings.cyruntime, cuda.bindings.cynvrtccuda.bindings.nvrtc, , , cuda.bindings.cyruntime, , cuda.bindings._bindings.cyruntime_ptdscuda.bindings.cynvrtc, cuda.bindings.nvrtc, scipy.spatial._qhullcuda.bindings._bindings.cyruntimecuda.bindings._bindings.cyruntime, cuda.bindings.cyruntime, cuda.bindings.nvrtc, , , cuda.bindings.nvrtccuda.bindings.cyruntime, cuda_utilscuda.bindings._bindings.cyruntimecuda.bindings._bindings.cyruntime, , , , cuda.bindings.cyruntimecuda.bindings._bindings.cyruntime_ptdscuda.bindings.runtime, , cuda.bindings._bindings.cyruntime_ptdscuda.bindings.runtime, cuda.bindings._bindings.cyruntime, , , cuda.bindings.runtimecuda.bindings._bindings.cyruntime, , , , cuda.bindings.nvrtccuda.bindings._bindings.cyruntimecuda.bindings.cyruntime, cuda.bindings.runtime, cuda.bindings.nvrtc, , cuda.bindings._bindings.cyruntime_ptdscuda.bindings.cyruntimecuda_utilscuda.bindings.cyruntime, , cuda.bindings.cyruntimecuda.bindings._bindings.cynvrtccuda.bindings.nvrtc, cuda.bindings.runtimecuda_utils, cuda.bindings._bindings.cyruntimecuda.bindings.nvrtc, , , cuda.bindings.cyruntime, cuda.bindings._bindings.cyruntime_ptds, scipy.spatial._voronoi, cuda.bindings.cyruntime, cuda.bindings.runtime, cuda.bindings._bindings.cyruntime_ptds, , cuda.bindings.runtime, cuda.bindings._bindings.cyruntime_ptds, , cuda.bindings.runtime, , cuda.bindings.runtimecuda.bindings._bindings.cyruntime, , cuda.bindings._bindings.cyruntime, cuda_utils, , , cuda.bindings.cyruntime__triton_launchercuda.bindings._bindings.cyruntime_ptdscuda_utils, , , cuda.bindings.cyruntime, cuda.bindings.cyruntime, cuda.bindings._bindings.cyruntime_ptdscuda.bindings.runtime, cuda.bindings.runtimecuda_utils, , cuda.bindings._bindings.cyruntime_ptdscuda.bindings._bindings.cyruntime, , , , cuda.bindings.runtime, cuda.bindings.runtimecuda.bindings.cynvrtccuda.bindings._bindings.cyruntime_ptds, cuda.bindings.cyruntime, cuda_utilscuda_utils (total: cuda.bindings.runtime__triton_launcher, __triton_launcher, cuda.bindings._bindings.cyruntimescipy.spatial._hausdorff, , , cuda.bindings._bindings.cyruntime, , , cuda.bindings._bindings.cyruntimecuda_utilscuda_utilscuda_utilscuda.bindings._bindings.cyruntimecuda.bindings._bindings.cyruntimecuda_utilscuda_utils, , cuda.bindings.cyruntimecuda.bindings.cyruntime, , cuda_utils111, , , cuda_utils (total: cuda.bindings._bindings.cyruntime, (total: cuda.bindings.runtime__triton_launchercuda.bindings.runtime__triton_launcher, cuda.bindings.runtimescipy.spatial._distance_wrap, , , , , cuda.bindings.cyruntime, cuda.bindings.cyruntime__triton_launcher, cuda.bindings.runtime, ), cuda.bindings.nvrtc__triton_launcher, , cuda.bindings._bindings.cyruntime, 111cuda_utils, , 111 (total: , (total: __triton_launchercuda.bindings.cyruntime, __triton_launcher, , __triton_launcher, cuda.bindings.cyruntimecuda.bindings.cyruntime, cuda.bindings.cyruntime (total: , cuda.bindings.runtime, __triton_launcher
> , (total: cuda.bindings.runtime, __triton_launcher__triton_launcher)__triton_launcher, cuda.bindings.cyruntime)111111 (total: cuda_utils, (total: cuda_utils__triton_launcher, (total: (total: cuda_utilsscipy.spatial.transform._rotation, , 111cuda.bindings.runtime, cuda.bindings.runtime (total: cuda_utils111, cuda.bindings._bindings.cyruntime_ptds (total: (total:
> (total: , cuda.bindings.cyruntime
> ), )111__triton_launcher111 (total: cuda.bindings.runtime111111, cuda.bindings.runtimecuda.bindings.runtime), , , cuda.bindings.runtime111, ), cuda_utils, 111111, )111cuda_utils),
> cuda.bindings.runtime
> , ))111))__triton_launcher
> , __triton_launcher, , scipy.spatial.transform._rigid_transformcuda_utils)cuda_utils
> cuda_utils__triton_launcher),
> cuda.bindings._bindings.cyruntime
>
> __triton_launcher, cuda.bindings.runtime
> )
>
> (total: cuda_utils (total: cuda_utilscuda_utils,
> (total: , 111cuda_utils, , (total: __triton_launcher, , , __triton_launcher
> (total: , 111111__triton_launcherscipy.optimize._direct)cuda.bindings.cyruntime__triton_launcher, 111 (total: __triton_launcher, __triton_launcher111cuda_utils), ) (total:
> (total: __triton_launcher)111 (total: , __triton_launcher (total: )
> __triton_launcher
> 111111 (total: ,
> )111cuda.bindings.runtime (total: 111
> (total: )), 111, setproctitle._setproctitle
> 111)111
>
> __triton_launcher)cuda_utils)
> ) (total:
>
> , __triton_launcher (total: 111)
> , core, cuda.bindings._bindings.cydriver, cuda.bindings.cydriver, cuda.bindings.driver, tvm_ffi.core, msgspec._core, cuda.bindings._bindings.cynvrtc, cuda.bindings.cynvrtc, cuda.bindings.nvrtc, cuda.bindings._bindings.cyruntime_ptds, cuda.bindings._bindings.cyruntime, cuda.bindings.cyruntime, cuda.bindings.runtime, cuda_utils, __triton_launcher (total: 111)
> ^CProcess Process-2:
> Traceback (most recent call last):
> File "/usr/lib/python3.12/multiprocessing/process.py", line 314, in _bootstrap
> self.run()
> File "/usr/lib/python3.12/multiprocessing/process.py", line 108, in run
> self._target(*self._args, **self._kwargs)
> File "/sgl-workspace/sglang/python/sglang/srt/managers/detokenizer_manager.py", line 320, in run_detokenizer_process
> manager.event_loop()
> File "/sgl-workspace/sglang/python/sglang/srt/managers/detokenizer_manager.py", line 115, in event_loop
> recv_obj = self.recv_from_scheduler.recv_pyobj()
> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> File "/usr/local/lib/python3.12/dist-packages/zmq/sugar/socket.py", line 989, in recv_pyobj
> msg = self.recv(flags)
> ^^^^^^^^^^^^^^^^
> File "zmq/backend/cython/_zmq.py", line 1218, in zmq.backend.cython._zmq.Socket.recv
> def recv(self, flags=0, copy: bint = True, track: bint = False):
> ^^^^^^^^^^^
> File "zmq/backend/cython/_zmq.py", line 1253, in zmq.backend.cython._zmq.Socket.recv
> return _recv_copy(self.handle, flags)
> ^^^^^^^^^^^
> File "zmq/backend/cython/_zmq.py", line 1408, in zmq.backend.cython._zmq._recv_copy
> _check_rc(rc)
> ^^^^^^^^^^^
> File "zmq/backend/cython/_zmq.py", line 179, in zmq.backend.cython._zmq._check_rc
> PyErr_CheckSignals()
> ^^^^^^^^^^^
> KeyboardInterrupt
> Traceback (most recent call last):
> File "", line 198, in _run_module_as_main
> File "", line 88, in _run_code
> File "/sgl-workspace/sglang/python/sglang/launch_server.py", line 27, in
> run_server(server_args)
> File "/sgl-workspace/sglang/python/sglang/launch_server.py", line 20, in run_server
> launch_server(server_args)
> File "/sgl-workspace/sglang/python/sglang/srt/entrypoints/http_server.py", line 1341, in launch_server
> _launch_subprocesses(server_args=server_args)
> File "/sgl-workspace/sglang/python/sglang/srt/entrypoints/engine.py", line 908, in _launch_subprocesses
> data = scheduler_pipe_readers[i].recv()
> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> File "/usr/lib/python3.12/multiprocessing/connection.py", line 250, in recv
> buf = self._recv_bytes()
> ^^^^^^^^^^^^^^^^^^
> File "/usr/lib/python3.12/multiprocessing/connection.py", line 430, in _recv_bytes
> buf = self._recv(4)
> ^^^^^^^^^^^^^
> File "/usr/lib/python3.12/multiprocessing/connection.py", line 395, in _recv
> chunk = read(handle, remaining)
> ^^^^^^^^^^^^^^^^^^^^^^^uct