kvcache-ai / kvcache-ai/ktransformers
Intel Consumer Level AVX-VNNI Support
- Dominant language
- Python
- Stars
- 19.5k
- Forks
- 1.6k
- Avg merge
- 19h 32m
- Merged PRs (30d)
- 27
Description
### Reminder
- [x] I have read the above rules and searched the existing issues.
### System Info
I am not sure.Does kt support the intel consumer level avx-vnni instructions whitch is 256 bit, it is different with server side avx-vnni which is 512 bit.Do you assess whether doing this is worthwhile? If so, I can do this work.I'm also trying to make the NPU, iGPU, and CPU work together on Intel processors.Can you give me some advice?
### Reproduction
```text
Put your message here.
```
### Others
_No response_
Contributor guide
Research direction
No source file, test, entry point, or reproducible case is identified. Start by clarifying whether ktransformers supports Intel consumer-level 256-bit AVX-VNNI separately from server-side 512-bit AVX-VNNI, then investigate the project’s existing CPU, NPU, and iGPU inference paths. Done should include an agreed feasibility assessment and a defined implementation scope.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- ai, performance
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100