kvcache-ai / kvcache-ai/ktransformers

[Bug] install and deploy successfully but no result or logs output

Open
#1,480 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
19.5k
Forks
1.6k
Avg merge
19h 32m
Merged PRs (30d)
27

Description

### 检查清单

- [x] 1. 我已经搜索过相关问题,但未能获得预期的帮助
- [x] 2. 该问题在最新版本中尚未修复
- [x] 3. 请注意,如果您提交的BUG相关 issue 缺少对应环境信息和最小可复现示例,我们将难以复现和定位问题,降低获得反馈的可能性
- [x] 4. 如果您提出的不是bug而是问题,请在讨论区发起讨论 https://github.com/kvcache-ai/ktransformers/discussions。否则该 issue 将被关闭
- [x] 5. 为方便社区交流,我将使用中文/英文或附上中文/英文翻译(如使用其他语言)。未附带翻译的非中文/英语内容可能会被关闭

### 问题描述

部署ktransformers0.3.2和unsloth/qwen3-235B-A22B-instruct-2507后终端日志显示正常,通过curl发起请求日志无输出,cpu利用率基本没有,等待半小时后也没有结果

### 复现步骤

Image如图,部署成功后发起请求,一直没有结果

启动命令如下:python /data/ktransformers/ktransformers/server/main.py \
--port 8080 \
--architectures Qwen3MoeForCausalLM \
--model_name Qwen3-235B-A22B-Instruct-2507 \
--model_path "/data/Qwen3-235B-A22B-Instruct-2507-HF" \
--gguf_path "/data/Qwen3-235B-A22B-Instruct-2507-GGUF" \
--optimize_config_path ktransformers/optimize/optimize_rules/Qwen3Moe-serve.yaml \
--cpu_infer 58 \
--temperature 0.3 \
--max_new_tokens 20000 \
--chunk_size 256 \
--max_batch_size 3 \
--backend_type balance_serve \
--cache_lens 60000

请求脚本:
curl -X POST http://localhost:8080/v1/chat/completions -H "accept: application/json" -H "Content-Type: application/json" -d '{ "messages": [ {"role": "user", "content": "上海有什么美丽的景点可以去旅游?"} ], "model": "Qwen3-30B-A3B", "temperature": 0.3, "top_p": 1.0, "stream": false}'

### 环境信息

操作系统:ubuntu22.04
cpu:INTEL(R) XEON(R) GOLD 6530 * 2
GPU: L20 * 2
DRAM: 515 GB DDR5

Contributor guide

Open the contributing guide

Research direction

Start with ktransformers/server/main.py and the Qwen3Moe-serve.yaml optimization configuration, then reproduce the documented launch command and curl request. Trace the request from the server entry point and capture where processing stops or logs disappear. Done means the supplied deployment accepts the request and returns a visible completion with corresponding server output.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend-api-design, machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.