kvcache-ai / kvcache-ai/ktransformers

卡在 loading model.layers.0.self_attn.k_norm.weight to cuda request to allocate mask for invalid number: Invalid argument

Open
#1,477 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
19.5k
Forks
1.6k
Avg merge
19h 32m
Merged PRs (30d)
27

Description

CPU :AMD9950X
GPU : 4090 * 2
操作系统 wsl ubuntu 22.04
运行模型Qwen3-30B-A22B 卡在卡在 loading model.layers.0.self_attn.k_norm.weight to cuda request to allocate mask for invalid number: Invalid argument

Injecting cache as default
Injecting lm_head as ktransformers.operators.linear . KTransformersLinear
loading model.embed_tokens.weight to cpu
loading model.layers.0.self_attn.q_norm.weight to cuda
loading model.layers.0.self_attn.k_norm.weight to cuda
request to allocate mask for invalid number: Invalid argument
request to allocate mask for invalid number: Invalid argument
request to allocate mask for invalid number: Invalid argument
request to allocate mask for invalid number: Invalid argument
request to allocate mask for invalid number: Invalid argument
request to allocate mask for invalid number: Invalid argument
request to allocate mask for invalid number: Invalid argument
request to allocate mask for invalid number: Invalid argument
request to allocate mask for invalid number: Invalid argument
request to allocate mask for invalid number: Invalid argument
request to allocate mask for invalid number: Invalid argument
request to allocate mask for invalid number: Invalid argument
^CTraceback (most recent call last):
File "/usr/local/ktransformers/ktransformers/server/main.py", line 122, in
main()
File "/usr/local/ktransformers/ktransformers/server/main.py", line 109, in main
create_interface(config=cfg, default_args=cfg)
File "/root/miniforge3/envs/ktransforms/lib/python3.12/site-packages/ktransformers/server/utils/create_interface.py", line 30, in create_interface
GlobalInterface.interface = BackendInterface(default_args)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/miniforge3/envs/ktransforms/lib/python3.12/site-packages/ktransformers/server/backend/interfaces/balance_serve.py", line 342, in __init__
kvcache_event.wait()
File "/root/miniforge3/envs/ktransforms/lib/python3.12/multiprocessing/synchronize.py", line 356, in wait
self._cond.wait(timeout)
File "/root/miniforge3/envs/ktransforms/lib/python3.12/multiprocessing/synchronize.py", line 268, in wait
return self._wait_semaphore.acquire(True, timeout)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
KeyboardInterrupt

请问各位大佬这个什么原因啊 怎么解决

Contributor guide

Open the contributing guide

Research direction

Reproduce the Qwen3-30B-A22B loading failure in the reported WSL Ubuntu 22.04 setup with two RTX 4090 GPUs. Start with ktransformers/server/main.py and the balance_serve.py path around BackendInterface and kvcache_event.wait(); determine why allocation repeatedly reports an invalid mask, then verify that model loading completes without the repeated error.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend, machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.