KTransformers Roadmap (2026 Q2)

# KTransformers Roadmap - 2026 Q2

## Focus
- Fine-tuning: Make LoRA SFT run on the exact same hardware as inference — if you can run it, you can fine-tune it.
- Consumer Hardware: Deliver the best MoE inference performance on consumer-grade x86 + NVIDIA GPU.
- Windows: Native Windows support with full heterogeneous performance advantage.
- Performance: VNNI instruction set optimization and AI SSD exploration.

## Fine-tuning Service @JimmyPeilinLi

**Goal: Lower the barrier for LoRA SFT to match inference — same hardware, same setup, zero extra cost.**

- Release fine-tuning service to main branch (basic version first, iterate later).
- Unify inference and SFT paths so both run on one set of hardware with shared model support, weight handling, and config.
- Support transformers v4 and v5, minimum 12G VRAM for 67B model. ✅

## Consumer Hardware Support

**Goal: Best-in-class MoE inference performance on consumer hardware.**

- Windows native C++ inference support — investigate JIT compiler Windows compatibility.
- Evaluate S1 inference backend migration to Windows as alternative path.
- Complete VNNI instruction set optimization.
- Investigate AI SSD performance bottleneck (slow mmap reads from disk).
- Promote AVX2 to first-class supported tier.
- Improve support quality and stability for RTX 30/40/50 series.

## Performance
- Optimize decode and prefill paths. @ouqingliang
- Optimize expert offloading and scheduling strategies. @ovowei
- Benchmark on representative AVX2 + RTX 30/40/50 configurations. @yyj6666667
- Focus on heterogeneous performance overhead in Windows path.
- Support nvfp4 and mxfp4 quantization formats (carried over from Q1).

## Coverage
- Fast support for important frontier MoE models.
- Improve support quality and completeness for major model families.

## Contribution / Maintenance
- Maintain CI/CD pipelines.
- Clearer documentation
- Establish recommended configurations for typical consumer hardware setups.
- Continue NUMA-aware optimization and CPU-GPU coordination improvements.

---
Any contribution is welcomed, please email yzwliam@126.com / ervinxie@qq.com if you want to join development WeChat group.

This is a broad roadmap covering fine-tuning, Windows support, inference performance, hardware, quantization, documentation, and CI/CD, with no files, tests, or entry points named. Start by selecting one workstream and clarifying its scope and acceptance criteria with maintainers; completion cannot be determined from this issue alone.

Written by the indexing model from the issue text.