[GLM-4.6 + KT-Kernel] TypeError in unquant.create_weights: w13_weight_n is None when enabling kt_ep_wrapper

I’m trying to run GLM-4.6 with KT-Kernel (CPU experts) via SGLang, using the official GLM-4.6 weights and the official `convert_cpu_weights.py` script. The model loads fine without KT-Kernel, but as soon as I enable `--kt-method` and `--kt-weight-path`, the scheduler fails during model initialization with:

```text
TypeError: empty() received an invalid combination of arguments - got (NoneType, int, int, dtype=torch.dtype)
```

The stack trace points to `unquant.py` and `kt_ep_wrapper.py`, where `w13_weight_n` appears to be `None`.

I’d like to confirm whether GLM-4.6 is expected to work with the KT-Kernel MoE EP backend right now, and if so, what the correct configuration / conversion steps are.

---

**Environment**

* Machine:

* CPU: Intel Xeon Platinum 8558P, 96 physical cores, AVX512, **no AMX**
* NUMA nodes: 1
* GPUs:

* 8 × NVIDIA H20-3e (≈ 144GB VRAM each)
* Driver: 550.163.01
* CUDA: 12.4
* OS: Linux x86\_64 (root\@10-0-0-14, Ubuntu-like environment)
* Python: 3.11 (Conda env `kt-kernel`)
* SGLang: installed from GitHub

```bash
git clone https://github.com/sgl-project/sglang.git
cd sglang
pip install -e "python[all]"
```
* KT-Kernel:

* Built in the same environment, with:

```bash
export CPUINFER_CPU_INSTRUCT=AVX512
export CPUINFER_ENABLE_AMX=OFF
# then build/install kt-kernel
```
* PyTorch / NCCL:

* SGLang reports: `sglang is using nccl==2.27.3` and `cudaDriverVersion 12040`

---

**Model & Weights**

* Base model: **GLM-4.6**, from ZhipuAI’s official release on ModelScope.
* GPU weights path:

* `/home/ubuntu/GLM-4.6`
* CPU weights conversion (official script):

```bash
python scripts/convert_cpu_weights.py \
--input-path /home/ubuntu/GLM-4.6 \
--input-type bf16 \
--output /root/GLM-4.6-cpu-weights \
--quant-method int8
```

So the CPU weights directory is:
`/root/GLM-4.6-cpu-weights`

---

**Launch Command (with KT-Kernel, where it fails)**

```bash
export CPUINFER_CPU_INSTRUCT=AVX512
export CPUINFER_ENABLE_AMX=OFF
export NCCL_DEBUG=INFO
export NCCL_P2P_DISABLE=1
export NCCL_IB_DISABLE=1
export NCCL_SOCKET_IFNAME=lo

CUDA_VISIBLE_DEVICES=0,1,2,3 \
python -m sglang.launch_server \
--model-path /home/ubuntu/GLM-4.6 \
--host 0.0.0.0 \
--port 8000 \
--tp 4 \
--mem-fraction-static 0.90 \
--context-length 128000 \
--attention-backend flashinfer \
--tool-call-parser glm45 \
--reasoning-parser glm45 \
--speculative-algorithm EAGLE \
--speculative-num-steps 3 \
--speculative-eagle-topk 1 \
--speculative-num-draft-tokens 4 \
--enable-metrics \
--served-model-name glm-4.6-inst1 \
\
--kt-method AMXINT8 \
--kt-weight-path /root/GLM-4.6-cpu-weights \
--kt-cpuinfer 48 \
--kt-threadpool-count 1 \
--kt-max-deferred-experts-per-token 2
```

---

**What works**

If I remove all `--kt-*` arguments, GLM-4.6 loads and serves correctly on the same machine with the same SGLang build, e.g.:

```bash
CUDA_VISIBLE_DEVICES=0,1,2,3 \
python -m sglang.launch_server \
--model-path /home/ubuntu/GLM-4.6 \
--host 0.0.0.0 \
--port 8000 \
--tp 4 \
--mem-fraction-static 0.90 \
--context-length 128000 \
--attention-backend flashinfer \
--tool-call-parser glm45 \
--reasoning-parser glm45 \
--speculative-algorithm EAGLE \
--speculative-num-steps 3 \
--speculative-eagle-topk 1 \
--speculative-num-draft-tokens 4 \
--enable-metrics \
--served-model-name glm-4.6-inst1
```

So the base GLM-4.6 + SGLang setup seems fine; the issue only appears once KT-Kernel MoE is enabled.

---

**Error Log (truncated)**

After torch distributed init and “Load weight begin”, I get the following exception from multiple TP ranks:

```text
[2025-11-15 14:24:11 TP0] Load weight begin. avail mem=139.01 GB
...

[2025-11-15 14:24:12 TP0] Scheduler hit an exception: Traceback (most recent call last):
File ".../sglang/srt/managers/scheduler.py", line 2709, in run_scheduler_process
scheduler = Scheduler(
File ".../sglang/srt/managers/scheduler.py", line 312, in __init__
self.tp_worker = TpModelWorker(
File ".../sglang/srt/managers/tp_worker.py", line 237, in __init__
self._model_runner = ModelRunner(
File ".../sglang/srt/model_executor/model_runner.py", line 323, in __init__
self.initialize(min_per_gpu_memory)
File ".../sglang/srt/model_executor/model_runner.py", line 409, in initialize
self.load_model()
File ".../sglang/srt/model_executor/model_runner.py", line 766, in load_model
self.model = get_model(
File ".../sglang/srt/model_loader/__init__.py", line 28, in get_model
return loader.load_model(
File ".../sglang/srt/model_loader/loader.py", line 594, in load_model
model = _initialize_model(
File ".../sglang/srt/model_loader/loader.py", line 262, in _initialize_model
return model_class(**kwargs)
File ".../sglang/srt/models/glm4_moe.py", line 952, in __init__
self.model = Glm4MoeModel(
File ".../sglang/srt/models/glm4_moe.py", line 843, in __init__
self.layers, self.start_layer, self.end_layer = make_layers(
File ".../sglang/srt/utils/common.py", line 577, in make_layers
+ get_offloader().wrap_modules(
File ".../sglang/srt/utils/offloader.py", line 36, in wrap_modules
return list(all_modules_generator)
File ".../sglang/srt/utils/common.py", line 579, in
layer_fn(idx=idx, prefix=add_prefix(idx, prefix))
File ".../sglang/srt/models/glm4_moe.py", line 845, in
lambda idx, prefix: Glm4MoeDecoderLayer(
File ".../sglang/srt/models/glm4_moe.py", line 664, in __init__
self.mlp = Glm4MoeSparseMoeBlock(
File ".../sglang/srt/models/glm4_moe.py", line 385, in __init__
self.experts = get_moe_impl_class(quant_config)(
File ".../sglang/srt/layers/moe/fused_moe_triton/layer.py", line 230, in __init__
self.quant_method.create_weights(
File ".../sglang/srt/layers/moe/kt_ep_wrapper.py", line 207, in create_weights
self.gpu_method.create_weights(
File ".../sglang/srt/layers/quantization/unquant.py", line 160, in create_weights
torch.empty(num_experts, w13_weight_n, w13_weight_k, dtype=params_dtype),
TypeError: empty() received an invalid combination of arguments - got (NoneType, int, int, dtype=torch.dtype), but expected one of:
* (tuple of ints size, *, tuple of names names, torch.memory_format memory_format = None, torch.dtype dtype = None, torch.layout layout = None, torch.device device = None, bool pin_memory = False, bool requires_grad = False)
* (tuple of ints size, *, torch.memory_format memory_format = None, Tensor out = None, torch.dtype dtype = None, torch.layout layout = None, torch.device device = None, bool pin_memory = False, bool requires_grad = False)
```

The same stack trace appears on TP1/TP2/TP3.

---

**Questions**

1. Is **GLM-4.6** currently supported with KT-Kernel MoE EP in SGLang (via `kt_ep_wrapper`)?
2. Is there a specific SGLang commit / branch and/or quantization config required for GLM-4.6 + KT-Kernel?
3. Does the `convert_cpu_weights.py` script need additional arguments for GLM-4.6, or are there known missing metadata fields (like `w13_weight_n`) for this model?
4. Any suggested workaround (e.g., disable certain MoE backend, change `kt-method`, or a temporary patch) to get GLM-4.6 running with CPU experts?

Happy to provide more logs (full `sglang` stdout, `config.json`, `quant_config`, etc.) if needed.