Issue: Reproducing Qwen3-MoE Ascend NPU tutorial leads to unexpected output

### Reminder

- [x] I have read the above rules and searched the existing issues.

### System Info

## Environment
- Server: Atlas 800I A2
- CPU: Kunpeng 920 (192 cores, NUMA 8 nodes)
- NPU: Ascend 910B3 × 8
- Mem：500GB
### CPU
```
Architecture: aarch64
CPU op-mode(s): 64-bit
Byte Order: Little Endian
CPU(s): 192
On-line CPU(s) list: 0-191
Vendor ID: HiSilicon
Model name: Kunpeng-920
Model: 0
Thread(s) per core: 1
Core(s) per cluster: 48
Socket(s): -
Cluster(s): 4
Stepping: 0x1
BogoMIPS: 200.00
Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma dcpop asimd
dp asimdfhm ssbs
Caches (sum of all):
L1d: 12 MiB (192 instances)
L1i: 12 MiB (192 instances)
L2: 96 MiB (192 instances)
L3: 192 MiB (8 instances)
NUMA:
NUMA node(s): 8
NUMA node0 CPU(s): 0-23
NUMA node1 CPU(s): 24-47
NUMA node2 CPU(s): 48-71
NUMA node3 CPU(s): 72-95
NUMA node4 CPU(s): 96-119
NUMA node5 CPU(s): 120-143
NUMA node6 CPU(s): 144-167
NUMA node7 CPU(s): 168-191
Vulnerabilities:
Itlb multihit: Not affected
L1tf: Not affected
Mds: Not affected
Meltdown: Not affected
Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Spectre v1: Mitigation; __user pointer sanitization
Spectre v2: Not affected
Srbds: Not affected
Tsx async abort: Not affected
```
### NPU
HDK : 25.3.RC1
```
+------------------------------------------------------------------------------------------------+
| npu-smi 25.3.rc1 Version: 25.3.rc1 |
+---------------------------+---------------+----------------------------------------------------+
| NPU Name | Health | Power(W) Temp(C) Hugepages-Usage(page)|
| Chip | Bus-Id | AICore(%) Memory-Usage(MB) HBM-Usage(MB) |
+===========================+===============+====================================================+
| 0 910B3 | OK | 92.4 34 0 / 0 |
| 0 | 0000:C1:00.0 | 0 0 / 0 3446 / 65536 |
+===========================+===============+====================================================+
| 1 910B3 | OK | 90.3 34 0 / 0 |
| 0 | 0000:C2:00.0 | 0 0 / 0 3403 / 65536 |
+===========================+===============+====================================================+
| 2 910B3 | OK | 94.5 32 0 / 0 |
| 0 | 0000:81:00.0 | 0 0 / 0 3203 / 65536 |
+===========================+===============+====================================================+
| 3 910B3 | OK | 95.7 34 0 / 0 |
| 0 | 0000:82:00.0 | 0 0 / 0 3203 / 65536 |
+===========================+===============+====================================================+
| 4 910B3 | OK | 94.7 37 0 / 0 |
| 0 | 0000:01:00.0 | 0 0 / 0 3203 / 65536 |
+===========================+===============+====================================================+
| 5 910B3 | OK | 95.2 38 0 / 0 |
| 0 | 0000:02:00.0 | 0 0 / 0 3203 / 65536 |
+===========================+===============+====================================================+
| 6 910B3 | OK | 93.1 37 0 / 0 |
| 0 | 0000:41:00.0 | 0 0 / 0 3203 / 65536 |
+===========================+===============+====================================================+
| 7 910B3 | OK | 93.1 37 0 / 0 |
| 0 | 0000:42:00.0 | 0 0 / 0 3203 / 65536 |
+===========================+===============+====================================================+
+---------------------------+---------------+----------------------------------------------------+
| NPU Chip | Process id | Process name | Process memory(MB) |
+===========================+===============+====================================================+
| No running processes found in NPU 0 |
+===========================+===============+====================================================+
| No running processes found in NPU 1 |
+===========================+===============+====================================================+
| No running processes found in NPU 2 |
+===========================+===============+====================================================+
| No running processes found in NPU 3 |
+===========================+===============+====================================================+
| No running processes found in NPU 4 |
+===========================+===============+====================================================+
| No running processes found in NPU 5 |
+===========================+===============+====================================================+
| No running processes found in NPU 6 |
+===========================+===============+====================================================+
| No running processes found in NPU 7 |
+===========================+===============+====================================================+
```
### NUMA structure
```
available: 8 nodes (0-7)
node 0 cpus: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23
node 0 size: 63876 MB
node 0 free: 36181 MB
node 1 cpus: 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47
node 1 size: 64508 MB
node 1 free: 45217 MB
node 2 cpus: 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71
node 2 size: 64508 MB
node 2 free: 49351 MB
node 3 cpus: 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95
node 3 size: 64508 MB
node 3 free: 48685 MB
node 4 cpus: 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119
node 4 size: 64508 MB
node 4 free: 48696 MB
node 5 cpus: 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143
node 5 size: 64508 MB
node 5 free: 46562 MB
node 6 cpus: 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167
node 6 size: 64508 MB
node 6 free: 45134 MB
node 7 cpus: 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191
node 7 size: 62449 MB
node 7 free: 46064 MB
node distances:
node 0 1 2 3 4 5 6 7
0: 10 11 24 25 24 25 24 25
1: 11 10 25 32 25 32 25 32
2: 24 25 10 11 24 25 24 25
3: 25 32 11 10 25 32 25 32
4: 24 25 24 25 10 11 24 25
5: 25 32 25 32 11 10 25 32
6: 24 25 24 25 24 25 10 11
7: 25 32 25 32 25 32 11 10
```

### Reproduction

### Guide
Followed https://github.com/kvcache-ai/ktransformers/blob/main/doc/zh/Qwen3-MoE_tutorial_zh_for_Ascend_NPU.md
### kt install
not use NUMA
```
# USE_BALANCE_SERVE=1 USE_NUMA=1 bash ./install.sh
USE_BALANCE_SERVE=1 bash ./install.sh
```
### run command
```
#!/bin/bash
export USE_MERGE=0
export INF_NAN_MODE_FORCE_DISABLE=1
export TASK_QUEUE_ENABLE=0
export RANK=0
export LOCAL_WORLD_SIZE=1
#export PROF_DECODE=1
#export PROF_PREFILL=1

source /usr/local/Ascend/ascend-toolkit/set_env.sh
source /usr/local/Ascend/nnal/atb/set_env.sh

python ktransformers/server/main.py \
--port 10002 \
--model_path /Qwen3/Qwen3-235B-A22B-q4km-w8a8/ \
--gguf_path /Qwen3/Qwen3-235B-A22B-q4km-w8a8/ \
--cpu_infer 24 \
--optimize_config_path ./ktransformers/optimize/optimize_rules/npu/Qwen3-Chat-300IA2-npu-serve.yaml \
--model_name Qwen3-235B-A22B \
--max_new_tokens 1024 \
--cache_lens 16384 \
--max_batch_size 4 \
--no-use_cuda_graph \
--tp 1 \
--backend_type balance_serve
```
### request
```
curl -X POST http://localhost:10002/v1/chat/completions \
-H "accept: application/json" \
-H "Content-Type: application/json" \
-d '{
"messages": [
{"role": "user", "content": "hello"}
],
"model": "Qwen3-235B-A22B",
"temperature": 0.3,
"top_p": 1.0,
"stream": false,
"max_new_tokens": 10
}'
```

### Others

## Result

The model produce only '\n' so I shut it down.

```
...
loading layer blk.93.120.ffn_up_exps to npu:0
loading layer blk.93.120.ffn_down_exps to npu:0
loading layer blk.93.121.ffn_gate_exps to npu:0
loading layer blk.93.121.ffn_up_exps to npu:0
loading layer blk.93.121.ffn_down_exps to npu:0
loading layer blk.93.122.ffn_gate_exps to npu:0
loading layer blk.93.122.ffn_up_exps to npu:0
loading layer blk.93.122.ffn_down_exps to npu:0
loading layer blk.93.123.ffn_gate_exps to npu:0
loading layer blk.93.123.ffn_up_exps to npu:0
loading layer blk.93.123.ffn_down_exps to npu:0
loading layer blk.93.124.ffn_gate_exps to npu:0
loading layer blk.93.124.ffn_up_exps to npu:0
loading layer blk.93.124.ffn_down_exps to npu:0
loading layer blk.93.125.ffn_gate_exps to npu:0
loading layer blk.93.125.ffn_up_exps to npu:0
loading layer blk.93.125.ffn_down_exps to npu:0
loading layer blk.93.126.ffn_gate_exps to npu:0
loading layer blk.93.126.ffn_up_exps to npu:0
loading layer blk.93.126.ffn_down_exps to npu:0
loading layer blk.93.127.ffn_gate_exps to npu:0
loading layer blk.93.127.ffn_up_exps to npu:0
loading layer blk.93.127.ffn_down_exps to npu:0
loading layer output_norm.weight to npu:0
Getting inference context from sched_client.
sched_rpc started with PID: 82551
Got inference context, sending it to subscribers.
Rebuilding kvcache
kv_cache loaded successfully.
[WARN] KNPUQwen3MoeForCausalLM does not use flashinfer wrapper on NPU, skip init_wrapper...
[WARN] the new ModelRunner and deviceId is npu:0
[WARMUP-NPU] skip warmup, eager mode!
/root/ktransformers/archive/ktransformers/server/utils/create_interface.py:33: ResourceWarning: unclosed file <_io.TextIOWrapper name='/root/.ktransformers/logs/rpc.log' mode='a' encoding='UTF-8'>
GlobalInterface.interface = BackendInterface(default_args, input_args)
2026-01-01 18:00:40,468 DEBUG /root/ktransformers/archive/ktransformers/server/backend/context_manager.py[23]: Creating Context Manager
2026-01-01 18:00:40,469 INFO /root/ktransformers/archive/ktransformers/server/main.py[31]: Creating SQL tables
2026-01-01 18:00:40,472 INFO /root/ktransformers/archive/ktransformers/server/api/openai/assistants/assistants.py[75]: Creating default assistant
INFO: Started server process [80731]
INFO: Waiting for application startup.
Queue Proxy Started
INFO: Application startup complete.
INFO: Uvicorn running on http://0.0.0.0:10002 (Press CTRL+C to quit)
/root/miniconda3/envs/kt-dev/lib/python3.11/site-packages/pydantic/main.py:528: UserWarning: Pydantic serializer warnings:
PydanticSerializationUnexpectedValue(Expected `list[dict[str, any]]` - serialized value may not be as expected [field_name='tool_calls', input_value={}, input_type=dict])
return self.__pydantic_serializer__.to_json(
2026-01-01 18:01:12,484 - INFO - flashinfer.jit: Prebuilt kernels not found, using JIT backend
found flashinfer
/root/miniconda3/envs/kt-dev/lib/python3.11/site-packages/torch_npu/contrib/transfer_to_npu.py:307: ImportWarning:
*************************************************************************************************************
The torch.Tensor.cuda and torch.nn.Module.cuda are replaced with torch.Tensor.npu and torch.nn.Module.npu now..
The torch.cuda.DoubleTensor is replaced with torch.npu.FloatTensor cause the double type is not supported now..
The backend in torch.distributed.init_process_group set to hccl now..
The torch.cuda.* and torch.cuda.amp.* are replaced with torch.npu.* and torch.npu.amp.* now..
The device parameters have been replaced with npu in the function below:
torch.logspace, torch.randint, torch.hann_window, torch.rand, torch.full_like, torch.ones_like, torch.rand_like, torch.randperm, torch.arange, torch.frombuffer, torch.normal, torch._empty_per_channel_affine_quantized, torch.empty_strided, torch.empty_like, torch.scalar_tensor, torch.tril_indices, torch.bartlett_window, torch.ones, torch.sparse_coo_tensor, torch.randn, torch.kaiser_window, torch.tensor, torch.triu_indices, torch.as_tensor, torch.zeros, torch.randint_like, torch.full, torch.eye, torch._sparse_csr_tensor_unsafe, torch.empty, torch._sparse_coo_tensor_unsafe, torch.blackman_window, torch.zeros_like, torch.range, torch.sparse_csr_tensor, torch.randn_like, torch.from_file, torch._cudnn_init_dropout_state, torch._empty_affine_quantized, torch.linspace, torch.hamming_window, torch.empty_quantized, torch._pin_memory, torch.load, torch.set_default_device, torch.Event, torch.get_device_module, torch.sparse_compressed_tensor, torch.Tensor.new_empty, torch.Tensor.new_empty_strided, torch.Tensor.new_full, torch.Tensor.new_ones, torch.Tensor.new_tensor, torch.Tensor.new_zeros, torch.Tensor.to, torch.Tensor.pin_memory, torch.nn.Module.to, torch.nn.Module.to_empty
*************************************************************************************************************

warnings.warn(msg, ImportWarning)
/root/miniconda3/envs/kt-dev/lib/python3.11/site-packages/torch_npu/contrib/transfer_to_npu.py:258: RuntimeWarning: torch.jit.script and torch.jit.script_method will be disabled by transfer_to_npu, which currently does not support them, if you need to enable them, please do not use transfer_to_npu.
warnings.warn(msg, RuntimeWarning)
/root/ktransformers/archive/ktransformers/server/api/ollama/completions.py:257: PydanticDeprecatedSince20: Support for class-based `config` is deprecated, use ConfigDict instead. Deprecated in Pydantic V2.0 to be removed in V3.0. See Pydantic V2 Migration Guide at https://errors.pydantic.dev/2.12/migration/
class OllamaShowResponse(BaseModel):
2026-01-01 18:01:17,380 DEBUG /root/ktransformers/archive/ktransformers/server/backend/interfaces/balance_serve.py[618]: get input ids of shape torch.Size([1, 9])
('Warning: torch.save with "_use_new_zipfile_serialization = False" is not recommended for npu tensor, which may bring unexpected errors and hopefully set "_use_new_zipfile_serialization = True"', 'if it is necessary to use this, please convert the npu tensor to cpu tensor for saving')
add query id: 1, batch.query_lengths: 9, batch_query_tokens: torch.Size([1033]), batch.block_indexes: tensor([0, 1, 2, 3, 4, 5, 6, 7, 8], dtype=torch.int32)
[PREFILL-QWEN3] k_full/v_full: torch.Size([1, 64, 9, 128]) torch.Size([1, 64, 9, 128])
[PREFILL-QWEN3] qTensor/kTensor/vTensor: torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128])
[PREFILL-QWEN3] attn_output(after o_proj): torch.Size([1, 9, 4096]) torch.float16
[PREFILL-QWEN3] k_full/v_full: torch.Size([1, 64, 9, 128]) torch.Size([1, 64, 9, 128])
[PREFILL-QWEN3] qTensor/kTensor/vTensor: torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128])
[PREFILL-QWEN3] attn_output(after o_proj): torch.Size([1, 9, 4096]) torch.float16
[PREFILL-QWEN3] k_full/v_full: torch.Size([1, 64, 9, 128]) torch.Size([1, 64, 9, 128])
[PREFILL-QWEN3] qTensor/kTensor/vTensor: torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128])
[PREFILL-QWEN3] attn_output(after o_proj): torch.Size([1, 9, 4096]) torch.float16
[PREFILL-QWEN3] k_full/v_full: torch.Size([1, 64, 9, 128]) torch.Size([1, 64, 9, 128])
[PREFILL-QWEN3] qTensor/kTensor/vTensor: torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128])
[PREFILL-QWEN3] attn_output(after o_proj): torch.Size([1, 9, 4096]) torch.float16
[PREFILL-QWEN3] k_full/v_full: torch.Size([1, 64, 9, 128]) torch.Size([1, 64, 9, 128])
[PREFILL-QWEN3] qTensor/kTensor/vTensor: torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128])
[PREFILL-QWEN3] attn_output(after o_proj): torch.Size([1, 9, 4096]) torch.float16
[PREFILL-QWEN3] k_full/v_full: torch.Size([1, 64, 9, 128]) torch.Size([1, 64, 9, 128])
[PREFILL-QWEN3] qTensor/kTensor/vTensor: torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128])
[PREFILL-QWEN3] attn_output(after o_proj): torch.Size([1, 9, 4096]) torch.float16
[PREFILL-QWEN3] k_full/v_full: torch.Size([1, 64, 9, 128]) torch.Size([1, 64, 9, 128])
[PREFILL-QWEN3] qTensor/kTensor/vTensor: torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128])
[PREFILL-QWEN3] attn_output(after o_proj): torch.Size([1, 9, 4096]) torch.float16
[PREFILL-QWEN3] k_full/v_full: torch.Size([1, 64, 9, 128]) torch.Size([1, 64, 9, 128])
[PREFILL-QWEN3] qTensor/kTensor/vTensor: torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128])
[PREFILL-QWEN3] attn_output(after o_proj): torch.Size([1, 9, 4096]) torch.float16
[PREFILL-QWEN3] k_full/v_full: torch.Size([1, 64, 9, 128]) torch.Size([1, 64, 9, 128])
[PREFILL-QWEN3] qTensor/kTensor/vTensor: torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128])
[PREFILL-QWEN3] attn_output(after o_proj): torch.Size([1, 9, 4096]) torch.float16
[PREFILL-QWEN3] k_full/v_full: torch.Size([1, 64, 9, 128]) torch.Size([1, 64, 9, 128])
[PREFILL-QWEN3] qTensor/kTensor/vTensor: torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128])
[PREFILL-QWEN3] attn_output(after o_proj): torch.Size([1, 9, 4096]) torch.float16
[PREFILL-QWEN3] k_full/v_full: torch.Size([1, 64, 9, 128]) torch.Size([1, 64, 9, 128])
[PREFILL-QWEN3] qTensor/kTensor/vTensor: torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128])
[PREFILL-QWEN3] attn_output(after o_proj): torch.Size([1, 9, 4096]) torch.float16
[PREFILL-QWEN3] k_full/v_full: torch.Size([1, 64, 9, 128]) torch.Size([1, 64, 9, 128])
[PREFILL-QWEN3] qTensor/kTensor/vTensor: torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128])
[PREFILL-QWEN3] attn_output(after o_proj): torch.Size([1, 9, 4096]) torch.float16
[PREFILL-QWEN3] k_full/v_full: torch.Size([1, 64, 9, 128]) torch.Size([1, 64, 9, 128])
[PREFILL-QWEN3] qTensor/kTensor/vTensor: torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128])
[PREFILL-QWEN3] attn_output(after o_proj): torch.Size([1, 9, 4096]) torch.float16
[PREFILL-QWEN3] k_full/v_full: torch.Size([1, 64, 9, 128]) torch.Size([1, 64, 9, 128])
[PREFILL-QWEN3] qTensor/kTensor/vTensor: torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128])
[PREFILL-QWEN3] attn_output(after o_proj): torch.Size([1, 9, 4096]) torch.float16
[PREFILL-QWEN3] k_full/v_full: torch.Size([1, 64, 9, 128]) torch.Size([1, 64, 9, 128])
[PREFILL-QWEN3] qTensor/kTensor/vTensor: torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128])
[PREFILL-QWEN3] attn_output(after o_proj): torch.Size([1, 9, 4096]) torch.float16
[PREFILL-QWEN3] k_full/v_full: torch.Size([1, 64, 9, 128]) torch.Size([1, 64, 9, 128])
[PREFILL-QWEN3] qTensor/kTensor/vTensor: torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128])
[PREFILL-QWEN3] attn_output(after o_proj): torch.Size([1, 9, 4096]) torch.float16
[PREFILL-QWEN3] k_full/v_full: torch.Size([1, 64, 9, 128]) torch.Size([1, 64, 9, 128])
[PREFILL-QWEN3] qTensor/kTensor/vTensor: torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128])
[PREFILL-QWEN3] attn_output(after o_proj): torch.Size([1, 9, 4096]) torch.float16
[PREFILL-QWEN3] k_full/v_full: torch.Size([1, 64, 9, 128]) torch.Size([1, 64, 9, 128])
[PREFILL-QWEN3] qTensor/kTensor/vTensor: torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128])
[PREFILL-QWEN3] attn_output(after o_proj): torch.Size([1, 9, 4096]) torch.float16
[PREFILL-QWEN3] k_full/v_full: torch.Size([1, 64, 9, 128]) torch.Size([1, 64, 9, 128])
[PREFILL-QWEN3] qTensor/kTensor/vTensor: torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128])
[PREFILL-QWEN3] attn_output(after o_proj): torch.Size([1, 9, 4096]) torch.float16
[PREFILL-QWEN3] k_full/v_full: torch.Size([1, 64, 9, 128]) torch.Size([1, 64, 9, 128])
[PREFILL-QWEN3] qTensor/kTensor/vTensor: torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128])
[PREFILL-QWEN3] attn_output(after o_proj): torch.Size([1, 9, 4096]) torch.float16
[PREFILL-QWEN3] k_full/v_full: torch.Size([1, 64, 9, 128]) torch.Size([1, 64, 9, 128])
[PREFILL-QWEN3] qTensor/kTensor/vTensor: torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128])
[PREFILL-QWEN3] attn_output(after o_proj): torch.Size([1, 9, 4096]) torch.float16
[PREFILL-QWEN3] k_full/v_full: torch.Size([1, 64, 9, 128]) torch.Size([1, 64, 9, 128])
[PREFILL-QWEN3] qTensor/kTensor/vTensor: torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128])
[PREFILL-QWEN3] attn_output(after o_proj): torch.Size([1, 9, 4096]) torch.float16
[PREFILL-QWEN3] k_full/v_full: torch.Size([1, 64, 9, 128]) torch.Size([1, 64, 9, 128])
[PREFILL-QWEN3] qTensor/kTensor/vTensor: torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128])
[PREFILL-QWEN3] attn_output(after o_proj): torch.Size([1, 9, 4096]) torch.float16
[PREFILL-QWEN3] k_full/v_full: torch.Size([1, 64, 9, 128]) torch.Size([1, 64, 9, 128])
[PREFILL-QWEN3] qTensor/kTensor/vTensor: torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128])
[PREFILL-QWEN3] attn_output(after o_proj): torch.Size([1, 9, 4096]) torch.float16
[PREFILL-QWEN3] k_full/v_full: torch.Size([1, 64, 9, 128]) torch.Size([1, 64, 9, 128])
[PREFILL-QWEN3] qTensor/kTensor/vTensor: torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128])
[PREFILL-QWEN3] attn_output(after o_proj): torch.Size([1, 9, 4096]) torch.float16
[PREFILL-QWEN3] k_full/v_full: torch.Size([1, 64, 9, 128]) torch.Size([1, 64, 9, 128])
[PREFILL-QWEN3] qTensor/kTensor/vTensor: torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128])
[PREFILL-QWEN3] attn_output(after o_proj): torch.Size([1, 9, 4096]) torch.float16
[PREFILL-QWEN3] k_full/v_full: torch.Size([1, 64, 9, 128]) torch.Size([1, 64, 9, 128])
[PREFILL-QWEN3] qTensor/kTensor/vTensor: torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128])
[PREFILL-QWEN3] attn_output(after o_proj): torch.Size([1, 9, 4096]) torch.float16
[PREFILL-QWEN3] k_full/v_full: torch.Size([1, 64, 9, 128]) torch.Size([1, 64, 9, 128])
[PREFILL-QWEN3] qTensor/kTensor/vTensor: torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128])
[PREFILL-QWEN3] attn_output(after o_proj): torch.Size([1, 9, 4096]) torch.float16
[PREFILL-QWEN3] k_full/v_full: torch.Size([1, 64, 9, 128]) torch.Size([1, 64, 9, 128])
[PREFILL-QWEN3] qTensor/kTensor/vTensor: torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128])
[PREFILL-QWEN3] attn_output(after o_proj): torch.Size([1, 9, 4096]) torch.float16
[PREFILL-QWEN3] k_full/v_full: torch.Size([1, 64, 9, 128]) torch.Size([1, 64, 9, 128])
[PREFILL-QWEN3] qTensor/kTensor/vTensor: torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128])
[PREFILL-QWEN3] attn_output(after o_proj): torch.Size([1, 9, 4096]) torch.float16
[PREFILL-QWEN3] k_full/v_full: torch.Size([1, 64, 9, 128]) torch.Size([1, 64, 9, 128])
[PREFILL-QWEN3] qTensor/kTensor/vTensor: torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128])
[PREFILL-QWEN3] attn_output(after o_proj): torch.Size([1, 9, 4096]) torch.float16
[PREFILL-QWEN3] k_full/v_full: torch.Size([1, 64, 9, 128]) torch.Size([1, 64, 9, 128])
[PREFILL-QWEN3] qTensor/kTensor/vTensor: torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128])
[PREFILL-QWEN3] attn_output(after o_proj): torch.Size([1, 9, 4096]) torch.float16
[PREFILL-QWEN3] k_full/v_full: torch.Size([1, 64, 9, 128]) torch.Size([1, 64, 9, 128])
[PREFILL-QWEN3] qTensor/kTensor/vTensor: torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128])
[PREFILL-QWEN3] attn_output(after o_proj): torch.Size([1, 9, 4096]) torch.float16
[PREFILL-QWEN3] k_full/v_full: torch.Size([1, 64, 9, 128]) torch.Size([1, 64, 9, 128])
[PREFILL-QWEN3] qTensor/kTensor/vTensor: torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128])
[PREFILL-QWEN3] attn_output(after o_proj): torch.Size([1, 9, 4096]) torch.float16
[PREFILL-QWEN3] k_full/v_full: torch.Size([1, 64, 9, 128]) torch.Size([1, 64, 9, 128])
[PREFILL-QWEN3] qTensor/kTensor/vTensor: torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128])
[PREFILL-QWEN3] attn_output(after o_proj): torch.Size([1, 9, 4096]) torch.float16
[PREFILL-QWEN3] k_full/v_full: torch.Size([1, 64, 9, 128]) torch.Size([1, 64, 9, 128])
[PREFILL-QWEN3] qTensor/kTensor/vTensor: torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128])
[PREFILL-QWEN3] attn_output(after o_proj): torch.Size([1, 9, 4096]) torch.float16
[PREFILL-QWEN3] k_full/v_full: torch.Size([1, 64, 9, 128]) torch.Size([1, 64, 9, 128])
[PREFILL-QWEN3] qTensor/kTensor/vTensor: torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128])
[PREFILL-QWEN3] attn_output(after o_proj): torch.Size([1, 9, 4096]) torch.float16
[PREFILL-QWEN3] k_full/v_full: torch.Size([1, 64, 9, 128]) torch.Size([1, 64, 9, 128])
[PREFILL-QWEN3] qTensor/kTensor/vTensor: torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128])
[PREFILL-QWEN3] attn_output(after o_proj): torch.Size([1, 9, 4096]) torch.float16
[PREFILL-QWEN3] k_full/v_full: torch.Size([1, 64, 9, 128]) torch.Size([1, 64, 9, 128])
[PREFILL-QWEN3] qTensor/kTensor/vTensor: torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128])
[PREFILL-QWEN3] attn_output(after o_proj): torch.Size([1, 9, 4096]) torch.float16
[PREFILL-QWEN3] k_full/v_full: torch.Size([1, 64, 9, 128]) torch.Size([1, 64, 9, 128])
[PREFILL-QWEN3] qTensor/kTensor/vTensor: torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128])
[PREFILL-QWEN3] attn_output(after o_proj): torch.Size([1, 9, 4096]) torch.float16
[PREFILL-QWEN3] k_full/v_full: torch.Size([1, 64, 9, 128]) torch.Size([1, 64, 9, 128])
[PREFILL-QWEN3] qTensor/kTensor/vTensor: torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128])
[PREFILL-QWEN3] attn_output(after o_proj): torch.Size([1, 9, 4096]) torch.float16
[PREFILL-QWEN3] k_full/v_full: torch.Size([1, 64, 9, 128]) torch.Size([1, 64, 9, 128])
[PREFILL-QWEN3] qTensor/kTensor/vTensor: torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128])
[PREFILL-QWEN3] attn_output(after o_proj): torch.Size([1, 9, 4096]) torch.float16
[PREFILL-QWEN3] k_full/v_full: torch.Size([1, 64, 9, 128]) torch.Size([1, 64, 9, 128])
[PREFILL-QWEN3] qTensor/kTensor/vTensor: torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128])
[PREFILL-QWEN3] attn_output(after o_proj): torch.Size([1, 9, 4096]) torch.float16
[PREFILL-QWEN3] k_full/v_full: torch.Size([1, 64, 9, 128]) torch.Size([1, 64, 9, 128])
[PREFILL-QWEN3] qTensor/kTensor/vTensor: torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128])
[PREFILL-QWEN3] attn_output(after o_proj): torch.Size([1, 9, 4096]) torch.float16
[PREFILL-QWEN3] k_full/v_full: torch.Size([1, 64, 9, 128]) torch.Size([1, 64, 9, 128])
[PREFILL-QWEN3] qTensor/kTensor/vTensor: torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128])
[PREFILL-QWEN3] attn_output(after o_proj): torch.Size([1, 9, 4096]) torch.float16
[PREFILL-QWEN3] k_full/v_full: torch.Size([1, 64, 9, 128]) torch.Size([1, 64, 9, 128])
[PREFILL-QWEN3] qTensor/kTensor/vTensor: torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128])
[PREFILL-QWEN3] attn_output(after o_proj): torch.Size([1, 9, 4096]) torch.float16
[PREFILL-QWEN3] k_full/v_full: torch.Size([1, 64, 9, 128]) torch.Size([1, 64, 9, 128])
[PREFILL-QWEN3] qTensor/kTensor/vTensor: torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128])
[PREFILL-QWEN3] attn_output(after o_proj): torch.Size([1, 9, 4096]) torch.float16
[PREFILL-QWEN3] k_full/v_full: torch.Size([1, 64, 9, 128]) torch.Size([1, 64, 9, 128])
[PREFILL-QWEN3] qTensor/kTensor/vTensor: torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128])
[PREFILL-QWEN3] attn_output(after o_proj): torch.Size([1, 9, 4096]) torch.float16
[PREFILL-QWEN3] k_full/v_full: torch.Size([1, 64, 9, 128]) torch.Size([1, 64, 9, 128])
[PREFILL-QWEN3] qTensor/kTensor/vTensor: torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128])
[PREFILL-QWEN3] attn_output(after o_proj): torch.Size([1, 9, 4096]) torch.float16
[PREFILL-QWEN3] k_full/v_full: torch.Size([1, 64, 9, 128]) torch.Size([1, 64, 9, 128])
[PREFILL-QWEN3] qTensor/kTensor/vTensor: torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128])
[PREFILL-QWEN3] attn_output(after o_proj): torch.Size([1, 9, 4096]) torch.float16
[PREFILL-QWEN3] k_full/v_full: torch.Size([1, 64, 9, 128]) torch.Size([1, 64, 9, 128])
[PREFILL-QWEN3] qTensor/kTensor/vTensor: torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128])
[PREFILL-QWEN3] attn_output(after o_proj): torch.Size([1, 9, 4096]) torch.float16
[PREFILL-QWEN3] k_full/v_full: torch.Size([1, 64, 9, 128]) torch.Size([1, 64, 9, 128])
[PREFILL-QWEN3] qTensor/kTensor/vTensor: torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128])
[PREFILL-QWEN3] attn_output(after o_proj): torch.Size([1, 9, 4096]) torch.float16
[PREFILL-QWEN3] k_full/v_full: torch.Size([1, 64, 9, 128]) torch.Size([1, 64, 9, 128])
[PREFILL-QWEN3] qTensor/kTensor/vTensor: torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128])
[PREFILL-QWEN3] attn_output(after o_proj): torch.Size([1, 9, 4096]) torch.float16
[PREFILL-QWEN3] k_full/v_full: torch.Size([1, 64, 9, 128]) torch.Size([1, 64, 9, 128])
[PREFILL-QWEN3] qTensor/kTensor/vTensor: torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128])
[PREFILL-QWEN3] attn_output(after o_proj): torch.Size([1, 9, 4096]) torch.float16
[PREFILL-QWEN3] k_full/v_full: torch.Size([1, 64, 9, 128]) torch.Size([1, 64, 9, 128])
[PREFILL-QWEN3] qTensor/kTensor/vTensor: torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128])
[PREFILL-QWEN3] attn_output(after o_proj): torch.Size([1, 9, 4096]) torch.float16
[PREFILL-QWEN3] k_full/v_full: torch.Size([1, 64, 9, 128]) torch.Size([1, 64, 9, 128])
[PREFILL-QWEN3] qTensor/kTensor/vTensor: torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128])
[PREFILL-QWEN3] attn_output(after o_proj): torch.Size([1, 9, 4096]) torch.float16
[PREFILL-QWEN3] k_full/v_full: torch.Size([1, 64, 9, 128]) torch.Size([1, 64, 9, 128])
[PREFILL-QWEN3] qTensor/kTensor/vTensor: torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128])
[PREFILL-QWEN3] attn_output(after o_proj): torch.Size([1, 9, 4096]) torch.float16
[PREFILL-QWEN3] k_full/v_full: torch.Size([1, 64, 9, 128]) torch.Size([1, 64, 9, 128])
[PREFILL-QWEN3] qTensor/kTensor/vTensor: torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128])
[PREFILL-QWEN3] attn_output(after o_proj): torch.Size([1, 9, 4096]) torch.float16
[PREFILL-QWEN3] k_full/v_full: torch.Size([1, 64, 9, 128]) torch.Size([1, 64, 9, 128])
[PREFILL-QWEN3] qTensor/kTensor/vTensor: torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128])
[PREFILL-QWEN3] attn_output(after o_proj): torch.Size([1, 9, 4096]) torch.float16
[PREFILL-QWEN3] k_full/v_full: torch.Size([1, 64, 9, 128]) torch.Size([1, 64, 9, 128])
[PREFILL-QWEN3] qTensor/kTensor/vTensor: torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128])
[PREFILL-QWEN3] attn_output(after o_proj): torch.Size([1, 9, 4096]) torch.float16
[PREFILL-QWEN3] k_full/v_full: torch.Size([1, 64, 9, 128]) torch.Size([1, 64, 9, 128])
[PREFILL-QWEN3] qTensor/kTensor/vTensor: torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128])
[PREFILL-QWEN3] attn_output(after o_proj): torch.Size([1, 9, 4096]) torch.float16
[PREFILL-QWEN3] k_full/v_full: torch.Size([1, 64, 9, 128]) torch.Size([1, 64, 9, 128])
[PREFILL-QWEN3] qTensor/kTensor/vTensor: torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128])
[PREFILL-QWEN3] attn_output(after o_proj): torch.Size([1, 9, 4096]) torch.float16
[PREFILL-QWEN3] k_full/v_full: torch.Size([1, 64, 9, 128]) torch.Size([1, 64, 9, 128])
[PREFILL-QWEN3] qTensor/kTensor/vTensor: torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128])
[PREFILL-QWEN3] attn_output(after o_proj): torch.Size([1, 9, 4096]) torch.float16
[PREFILL-QWEN3] k_full/v_full: torch.Size([1, 64, 9, 128]) torch.Size([1, 64, 9, 128])
[PREFILL-QWEN3] qTensor/kTensor/vTensor: torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128])
[PREFILL-QWEN3] attn_output(after o_proj): torch.Size([1, 9, 4096]) torch.float16
[PREFILL-QWEN3] k_full/v_full: torch.Size([1, 64, 9, 128]) torch.Size([1, 64, 9, 128])
[PREFILL-QWEN3] qTensor/kTensor/vTensor: torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128])
[PREFILL-QWEN3] attn_output(after o_proj): torch.Size([1, 9, 4096]) torch.float16
[PREFILL-QWEN3] k_full/v_full: torch.Size([1, 64, 9, 128]) torch.Size([1, 64, 9, 128])
[PREFILL-QWEN3] qTensor/kTensor/vTensor: torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128])
[PREFILL-QWEN3] attn_output(after o_proj): torch.Size([1, 9, 4096]) torch.float16
[PREFILL-QWEN3] k_full/v_full: torch.Size([1, 64, 9, 128]) torch.Size([1, 64, 9, 128])
[PREFILL-QWEN3] qTensor/kTensor/vTensor: torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128])
[PREFILL-QWEN3] attn_output(after o_proj): torch.Size([1, 9, 4096]) torch.float16
[PREFILL-QWEN3] k_full/v_full: torch.Size([1, 64, 9, 128]) torch.Size([1, 64, 9, 128])
[PREFILL-QWEN3] qTensor/kTensor/vTensor: torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128])
[PREFILL-QWEN3] attn_output(after o_proj): torch.Size([1, 9, 4096]) torch.float16
[PREFILL-QWEN3] k_full/v_full: torch.Size([1, 64, 9, 128]) torch.Size([1, 64, 9, 128])
[PREFILL-QWEN3] qTensor/kTensor/vTensor: torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128])
[PREFILL-QWEN3] attn_output(after o_proj): torch.Size([1, 9, 4096]) torch.float16
[PREFILL-QWEN3] k_full/v_full: torch.Size([1, 64, 9, 128]) torch.Size([1, 64, 9, 128])
[PREFILL-QWEN3] qTensor/kTensor/vTensor: torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128])
[PREFILL-QWEN3] attn_output(after o_proj): torch.Size([1, 9, 4096]) torch.float16
[PREFILL-QWEN3] k_full/v_full: torch.Size([1, 64, 9, 128]) torch.Size([1, 64, 9, 128])
[PREFILL-QWEN3] qTensor/kTensor/vTensor: torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128])
[PREFILL-QWEN3] attn_output(after o_proj): torch.Size([1, 9, 4096]) torch.float16
[PREFILL-QWEN3] k_full/v_full: torch.Size([1, 64, 9, 128]) torch.Size([1, 64, 9, 128])
[PREFILL-QWEN3] qTensor/kTensor/vTensor: torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128])
[PREFILL-QWEN3] attn_output(after o_proj): torch.Size([1, 9, 4096]) torch.float16
[PREFILL-QWEN3] k_full/v_full: torch.Size([1, 64, 9, 128]) torch.Size([1, 64, 9, 128])
[PREFILL-QWEN3] qTensor/kTensor/vTensor: torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128])
[PREFILL-QWEN3] attn_output(after o_proj): torch.Size([1, 9, 4096]) torch.float16
[PREFILL-QWEN3] k_full/v_full: torch.Size([1, 64, 9, 128]) torch.Size([1, 64, 9, 128])
[PREFILL-QWEN3] qTensor/kTensor/vTensor: torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128])
[PREFILL-QWEN3] attn_output(after o_proj): torch.Size([1, 9, 4096]) torch.float16
[PREFILL-QWEN3] k_full/v_full: torch.Size([1, 64, 9, 128]) torch.Size([1, 64, 9, 128])
[PREFILL-QWEN3] qTensor/kTensor/vTensor: torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128])
[PREFILL-QWEN3] attn_output(after o_proj): torch.Size([1, 9, 4096]) torch.float16
[PREFILL-QWEN3] k_full/v_full: torch.Size([1, 64, 9, 128]) torch.Size([1, 64, 9, 128])
[PREFILL-QWEN3] qTensor/kTensor/vTensor: torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128])
[PREFILL-QWEN3] attn_output(after o_proj): torch.Size([1, 9, 4096]) torch.float16
[PREFILL-QWEN3] k_full/v_full: torch.Size([1, 64, 9, 128]) torch.Size([1, 64, 9, 128])
[PREFILL-QWEN3] qTensor/kTensor/vTensor: torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128])
[PREFILL-QWEN3] attn_output(after o_proj): torch.Size([1, 9, 4096]) torch.float16
[PREFILL-QWEN3] k_full/v_full: torch.Size([1, 64, 9, 128]) torch.Size([1, 64, 9, 128])
[PREFILL-QWEN3] qTensor/kTensor/vTensor: torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128])
[PREFILL-QWEN3] attn_output(after o_proj): torch.Size([1, 9, 4096]) torch.float16
[PREFILL-QWEN3] k_full/v_full: torch.Size([1, 64, 9, 128]) torch.Size([1, 64, 9, 128])
[PREFILL-QWEN3] qTensor/kTensor/vTensor: torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128])
[PREFILL-QWEN3] attn_output(after o_proj): torch.Size([1, 9, 4096]) torch.float16
[PREFILL-QWEN3] k_full/v_full: torch.Size([1, 64, 9, 128]) torch.Size([1, 64, 9, 128])
[PREFILL-QWEN3] qTensor/kTensor/vTensor: torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128])
[PREFILL-QWEN3] attn_output(after o_proj): torch.Size([1, 9, 4096]) torch.float16
[PREFILL-QWEN3] k_full/v_full: torch.Size([1, 64, 9, 128]) torch.Size([1, 64, 9, 128])
[PREFILL-QWEN3] qTensor/kTensor/vTensor: torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128])
[PREFILL-QWEN3] attn_output(after o_proj): torch.Size([1, 9, 4096]) torch.float16
[PREFILL-QWEN3] k_full/v_full: torch.Size([1, 64, 9, 128]) torch.Size([1, 64, 9, 128])
[PREFILL-QWEN3] qTensor/kTensor/vTensor: torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128])
[PREFILL-QWEN3] attn_output(after o_proj): torch.Size([1, 9, 4096]) torch.float16
[PREFILL-QWEN3] k_full/v_full: torch.Size([1, 64, 9, 128]) torch.Size([1, 64, 9, 128])
[PREFILL-QWEN3] qTensor/kTensor/vTensor: torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128])
[PREFILL-QWEN3] attn_output(after o_proj): torch.Size([1, 9, 4096]) torch.float16
[PREFILL-QWEN3] k_full/v_full: torch.Size([1, 64, 9, 128]) torch.Size([1, 64, 9, 128])
[PREFILL-QWEN3] qTensor/kTensor/vTensor: torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128])
[PREFILL-QWEN3] attn_output(after o_proj): torch.Size([1, 9, 4096]) torch.float16
[PREFILL-QWEN3] k_full/v_full: torch.Size([1, 64, 9, 128]) torch.Size([1, 64, 9, 128])
[PREFILL-QWEN3] qTensor/kTensor/vTensor: torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128])
[PREFILL-QWEN3] attn_output(after o_proj): torch.Size([1, 9, 4096]) torch.float16
[PREFILL-QWEN3] k_full/v_full: torch.Size([1, 64, 9, 128]) torch.Size([1, 64, 9, 128])
[PREFILL-QWEN3] qTensor/kTensor/vTensor: torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128])
[PREFILL-QWEN3] attn_output(after o_proj): torch.Size([1, 9, 4096]) torch.float16
[PREFILL-QWEN3] k_full/v_full: torch.Size([1, 64, 9, 128]) torch.Size([1, 64, 9, 128])
[PREFILL-QWEN3] qTensor/kTensor/vTensor: torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128])
[PREFILL-QWEN3] attn_output(after o_proj): torch.Size([1, 9, 4096]) torch.float16
[PREFILL-QWEN3] k_full/v_full: torch.Size([1, 64, 9, 128]) torch.Size([1, 64, 9, 128])
[PREFILL-QWEN3] qTensor/kTensor/vTensor: torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128])
[PREFILL-QWEN3] attn_output(after o_proj): torch.Size([1, 9, 4096]) torch.float16
[PREFILL-QWEN3] k_full/v_full: torch.Size([1, 64, 9, 128]) torch.Size([1, 64, 9, 128])
[PREFILL-QWEN3] qTensor/kTensor/vTensor: torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128])
[PREFILL-QWEN3] attn_output(after o_proj): torch.Size([1, 9, 4096]) torch.float16
[PREFILL-QWEN3] k_full/v_full: torch.Size([1, 64, 9, 128]) torch.Size([1, 64, 9, 128])
[PREFILL-QWEN3] qTensor/kTensor/vTensor: torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128])
[PREFILL-QWEN3] attn_output(after o_proj): torch.Size([1, 9, 4096]) torch.float16
[PREFILL-QWEN3] k_full/v_full: torch.Size([1, 64, 9, 128]) torch.Size([1, 64, 9, 128])
[PREFILL-QWEN3] qTensor/kTensor/vTensor: torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128])
[PREFILL-QWEN3] attn_output(after o_proj): torch.Size([1, 9, 4096]) torch.float16
[PREFILL-QWEN3] k_full/v_full: torch.Size([1, 64, 9, 128]) torch.Size([1, 64, 9, 128])
[PREFILL-QWEN3] qTensor/kTensor/vTensor: torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128])
[PREFILL-QWEN3] attn_output(after o_proj): torch.Size([1, 9, 4096]) torch.float16
[PREFILL-QWEN3] k_full/v_full: torch.Size([1, 64, 9, 128]) torch.Size([1, 64, 9, 128])
[PREFILL-QWEN3] qTensor/kTensor/vTensor: torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128])
[PREFILL-QWEN3] attn_output(after o_proj): torch.Size([1, 9, 4096]) torch.float16
[PREFILL-QWEN3] k_full/v_full: torch.Size([1, 64, 9, 128]) torch.Size([1, 64, 9, 128])
[PREFILL-QWEN3] qTensor/kTensor/vTensor: torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128]) torch.Size([1, 9, 64, 128])
[PREFILL-QWEN3] attn_output(after o_proj): torch.Size([1, 9, 4096]) torch.float16
/root/miniconda3/envs/kt-dev/lib/python3.11/site-packages/torch_npu/utils/storage.py:41: UserWarning: TypedStorage is deprecated. It will be removed in the future and UntypedStorage will be the only storage class. This should only matter to you if you are using storages directly. To access UntypedStorage directly, use tensor.untyped_storage() instead of tensor.storage()
if self.device.type != 'cpu':

./run.sh: line 25: 80731 Killed python ktransformers/server/main.py --port 10002 --model_path /Qwen3/Qwen3-235B-A22B-q4km-w8a8/ --gguf_path /Qwen3/Qwen3-235B-A22B-q4km-w8a8/ --cpu_infer 24 --optimize_config_path ./ktransformers/optimize/optimize_rules/npu/Qwen3-Chat-300IA2-npu-serve.yaml --model_name Qwen3-235B-A22B --max_new_tokens 1024 --cache_lens 16384 --max_batch_size 4 --no-use_cuda_graph --tp 1 --backend_type balance_serve
```

Start with doc/zh/Qwen3-MoE_tutorial_zh_for_Ascend_NPU.md, the command invoking ktransformers/server/main.py, and ktransformers/optimize/optimize_rules/npu/Qwen3-Chat-300IA2-npu-serve.yaml. Reproduce the curl request in the reported environment and trace the server output to determine why the response is only a newline; done means the tutorial produces a normal response or the failure is documented with a confirmed cause.

Written by the indexing model from the issue text.