kvcache-ai / kvcache-ai/ktransformers

DeepSeekV3.2 乱码

Open
#1,650 2 comments 0 reactions 1 assignee Claimed by @ovowei View on GitHub
bug
Dominant language
Python
Stars
19.5k
Forks
1.6k
Avg merge
19h 32m
Merged PRs (30d)
27

Description

### Reminder

- [x] I have read the above rules and searched the existing issues.

### System Info

Image

`AMX_METHOD=AMXINT8 SGLANG_ENABLE_JIT_DEEPGEMM=0 PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True python -m sglang.launch_server --host 0.0.0.0 --port 60000 --model /mnt/ktd/v3.2exp/ --kt-weight-path ~/v3.2cpu/ --kt-cpuinfer 116 --kt-threadpool-count 2 --kt-num-gpu-experts 5 --kt-method AMXINT8 --attention-backend flashinfer --trust-remote-code --mem-fraction-static 0.5 --chunked-prefill-size 1024 --max-running-requests 4 --enable-mixed-chunk --tensor-parallel-size 1 --enable-p2p-check --disable-shared-experts-fusion --served-model-name kimi_k2 --tool-call-parser deepseekv3 --kt-max-deferred-experts-per-token 7`

运行命令是这个。

Image

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.