kvcache-ai / kvcache-ai/ktransformers

[Feature] Add support for OpenAI's new open-source models gpt-oss-120b and gpt-oss-20b

Open
#1,472 2 comments 6 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
19.5k
Forks
1.6k
Avg merge
19h 32m
Merged PRs (30d)
27

Description

### Checklist

- [x] 1. If the issue you raised is not a feature but a question, please raise a discussion at https://github.com/kvcache-ai/ktransformers/discussions. Otherwise, it will be closed.
- [x] 2. To help the community, I will use Chinese/English or attach an Chinese/English translation if using another language. Non-English/Chinese content without translation may be closed.

### Motivation

- MoE Architecture Optimization: Both gpt-oss models use MoE with sparse activation (5.1B/3.6B active params) - perfectly suited for ktransformers' CPU-GPU heterogeneous computing and expert routing optimization.
- These are OpenAI's first open models in years with huge adoption potential - early support would position ktransformers as the go-to optimization framework.

### Related resources

OpenAI blog post: https://openai.com/index/introducing-gpt-oss/
Model cards: https://openai.com/index/gpt-oss-model-card/

Contributor guide

Open the contributing guide

Research direction

Start by reviewing the repository's existing model-support entry points and the linked OpenAI blog post and model card. Done means ktransformers supports both gpt-oss-120b and gpt-oss-20b, including their MoE architecture and heterogeneous CPU-GPU execution needs.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.