kvcache-ai / kvcache-ai/ktransformers
[Bug] Cannot run DeepSeek-TNG-R1T2-Chimera IQ1_S UD
- Dominant language
- Python
- Stars
- 19.5k
- Forks
- 1.6k
- Avg merge
- 19h 32m
- Merged PRs (30d)
- 27
Description
### 检查清单
- [x] 1. 我已经搜索过相关问题,但未能获得预期的帮助
- [x] 2. 该问题在最新版本中尚未修复
- [x] 3. 请注意,如果您提交的BUG相关 issue 缺少对应环境信息和最小可复现示例,我们将难以复现和定位问题,降低获得反馈的可能性
- [x] 4. 如果您提出的不是bug而是问题,请在讨论区发起讨论 https://github.com/kvcache-ai/ktransformers/discussions。否则该 issue 将被关闭
- [x] 5. 为方便社区交流,我将使用中文/英文或附上中文/英文翻译(如使用其他语言)。未附带翻译的非中文/英语内容可能会被关闭
### 问题描述
When trying to run DeepSeek-TNG-R1T2-Chimera-UD-IQ1_S, following error occurs:
NotImplementedError: ggml_type 18 not implemented
According to https://kvcache-ai.github.io/ktransformers/en/install.html#support-quantize-format
IQ1_S should be supported correctly.
### 复现步骤
ktransformers --gguf_path /models/ --model_path tngtech/DeepSeek-TNG-R1T2-Chimera --model_name unsloth/DeepSeek-TNG-R1T2-Chimera-GGUF --cache_q4 true --temperature 0.6 --top_p 0.95 --force_think --host 0.0.0.0 --port 10002 --optimize_config_path /models/DeepSeek-V3-Chat.yaml
Model: DeepSeek-TNG-R1T2-Chimera
### 环境信息
Dockerfile: [approachingai/ktransformers:latest-AVX2]
GPU: NVIDIA A4000 16G
CPU: AMD EPYC 7773X
Contributor guide
Research direction
Start by reproducing the command with the DeepSeek-TNG-R1T2-Chimera IQ1_S model in the approachingai/ktransformers:latest-AVX2 environment. Trace the ktransformers CLI path that handles the model until ggml_type 18 raises NotImplementedError, then compare it with the documented IQ1_S support; done means the supplied command runs without that error.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- docker, python
- Domain
- cli, machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 42/100