kvcache-ai / kvcache-ai/ktransformers

Intel Consumer Level AVX-VNNI Support

Open
#1,897 7 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
19.5k
Forks
1.6k
Avg merge
19h 32m
Merged PRs (30d)
27

Description

### Reminder

- [x] I have read the above rules and searched the existing issues.

### System Info

I am not sure.Does kt support the intel consumer level avx-vnni instructions whitch is 256 bit, it is different with server side avx-vnni which is 512 bit.Do you assess whether doing this is worthwhile? If so, I can do this work.I'm also trying to make the NPU, iGPU, and CPU work together on Intel processors.Can you give me some advice?

### Reproduction

```text
Put your message here.
```

### Others

_No response_

Contributor guide

Open the contributing guide

Research direction

No source file, test, entry point, or reproducible case is identified. Start by clarifying whether ktransformers supports Intel consumer-level 256-bit AVX-VNNI separately from server-side 512-bit AVX-VNNI, then investigate the project’s existing CPU, NPU, and iGPU inference paths. Done should include an agreed feasibility assessment and a defined implementation scope.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
ai, performance
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.