kvcache-ai / kvcache-ai/ktransformers
[Bug] AssertionError: [Smallthinker config sanitizer] rope_layout must have the same length as num_hidden_layers
- Dominant language
- Python
- Stars
- 19.5k
- Forks
- 1.6k
- Avg merge
- 19h 32m
- Merged PRs (30d)
- 27
Description
### Checklist
- [x] 1. I have searched related issues but cannot get the expected help.
- [ ] 2. The bug has not been fixed in the latest version.
- [ ] 3. Please note that if the bug-related issue you submitted lacks corresponding environment info and a minimal reproducible demo, it will be challenging for us to reproduce and resolve the issue, reducing the likelihood of receiving feedback.
- [ ] 4. If the issue you raised is not a bug but a question, please raise a discussion at https://github.com/kvcache-ai/ktransformers/discussions. Otherwise, it will be closed.
- [ ] 5. To help the community, I will use Chinese/English or attach an Chinese/English translation if using another language. Non-Chinese/English content without translation may be closed.
### Describe the bug
python ktransformers/server/main.py \
--model_path /home/dhr/projects/dhr/kf/ktransformers/deepseek_v3 \
--gguf_path /home/dhr/projects/dhr/kf/ktransformers/deepseek_v3_q4_gguf \
--cpu_infer 15 \
--optimize_config_path ktransformers/optimize/optimize_rules/DeepSeek-V3-Chat-serve.yaml \
--port 10002 \
--chunk_size 256 \
--max_new_tokens 1024 \
--max_batch_size 4 \
--port 10002 \
--cache_lens 3000 \
--backend_type balance_serve
是我模型下错了吗?但是我使用local_chat的时候是可行的
python3 -m ktransformers.local_chat \
--model_path "/home/dhr/projects/dhr/kf/ktransformers/deepseek_v3" \
--gguf_path "/home/dhr/projects/dhr/kf/ktransformers/deepseek_v3_q4_gguf"
### Reproduction
deepseek-v3
### Environment
按照install.md中的步骤创建的环境
Contributor guide
Research direction
Start with ktransformers/server/main.py and compare its DeepSeek-V3 startup path with python3 -m ktransformers.local_chat, using the commands in the report. Review ktransformers/optimize/optimize_rules/DeepSeek-V3-Chat-serve.yaml and the install.md environment steps. Done means the server reproduces no rope_layout assertion, or the missing compatibility details are documented with a minimal reproduction.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- ai, backend
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100