kvcache-ai / kvcache-ai/ktransformers

[Bug] Cannot run DeepSeek-TNG-R1T2-Chimera IQ1_S UD

Open
#1,444 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
19.5k
Forks
1.6k
Avg merge
19h 32m
Merged PRs (30d)
27

Description

### 检查清单

- [x] 1. 我已经搜索过相关问题,但未能获得预期的帮助
- [x] 2. 该问题在最新版本中尚未修复
- [x] 3. 请注意,如果您提交的BUG相关 issue 缺少对应环境信息和最小可复现示例,我们将难以复现和定位问题,降低获得反馈的可能性
- [x] 4. 如果您提出的不是bug而是问题,请在讨论区发起讨论 https://github.com/kvcache-ai/ktransformers/discussions。否则该 issue 将被关闭
- [x] 5. 为方便社区交流,我将使用中文/英文或附上中文/英文翻译(如使用其他语言)。未附带翻译的非中文/英语内容可能会被关闭

### 问题描述

When trying to run DeepSeek-TNG-R1T2-Chimera-UD-IQ1_S, following error occurs:
NotImplementedError: ggml_type 18 not implemented

According to https://kvcache-ai.github.io/ktransformers/en/install.html#support-quantize-format
IQ1_S should be supported correctly.

### 复现步骤

ktransformers --gguf_path /models/ --model_path tngtech/DeepSeek-TNG-R1T2-Chimera --model_name unsloth/DeepSeek-TNG-R1T2-Chimera-GGUF --cache_q4 true --temperature 0.6 --top_p 0.95 --force_think --host 0.0.0.0 --port 10002 --optimize_config_path /models/DeepSeek-V3-Chat.yaml

Model: DeepSeek-TNG-R1T2-Chimera

### 环境信息

Dockerfile: [approachingai/ktransformers:latest-AVX2]
GPU: NVIDIA A4000 16G
CPU: AMD EPYC 7773X

Contributor guide

Open the contributing guide

Research direction

Start by reproducing the command with the DeepSeek-TNG-R1T2-Chimera IQ1_S model in the approachingai/ktransformers:latest-AVX2 environment. Trace the ktransformers CLI path that handles the model until ggml_type 18 raises NotImplementedError, then compare it with the documented IQ1_S support; done means the supplied command runs without that error.

Written by the indexing model from the issue text.

Assessment

Tech stack
docker, python
Domain
cli, machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.