intel / intel/llm-scaler

Will we ever get training tools like axolotl for gpt-oss type models. Because 8 B60's RIP through distill

Open
#181 6 comments 1 reaction 1 assignee Claimed by @liu-shaojun View on GitHub
enhancement user issue
Dominant language
C++
Stars
529
Forks
80
Avg merge
9h 7m
Merged PRs (30d)
38

Description

I ran a distill off gpt-oss-120b today, at 32 concurrent I was going over 750 t/s at like 85 watts per GPU.

Then disappointment set it, I have to dequant the model and run on CPU and system ram for training gpt-oss-20b.

I am training the smaller model for my business on my business domains.

distill makes nice screenshots though. See attached.

Image
Image

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.