kvcache-ai / kvcache-ai/ktransformers

关于是整个模型量化还是CPU量化

Open
#1,896 1 comment 0 reactions 1 assignee Claimed by @chenht2022 View on GitHub
Dominant language
Python
Stars
19.5k
Forks
1.6k
Avg merge
19h 32m
Merged PRs (30d)
27

Description

### Reminder

- [x] I have read the above rules and searched the existing issues.

### System Info

我想知道论文里提到的deepseek v3的INT4的实验,指的是把整个模型参数量化到INT4,还是指的是只量化了CPU端的参数

### Reproduction

```text
Put your message here.
```

### Others

_No response_

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.