AlexsJones / AlexsJones/llmfit

fix(fit): prefer native MXFP4 quant path for gpt-oss-class MoE models

未关闭
#973 1 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看
Efficiency enhancement
主要语言
Rust
星标
36.3k
派生
2.3k
平均合并
2 天 18 小时
30 天内合并 PR
85

描述

## Context

PR #971 makes `plan --quant Q4_K_M --profile ryzen-ai-max-plus-395 openai/gpt-oss-120b` report **~50 tok/s** (matching the #969 measured reference). Dynamic `fit` still picks **Q8_0** on a 128 GB profile and estimates **~23.5 tok/s**, because gpt-oss ships **MXFP4-native** weights and the Tier-1 scalable term prices them at `quant_bpp("Q8_0")`.

## Ask

- Teach quant selection / bpp that `gpt_oss` (and similar) prefer native MXFP4 / low-precision quants when available
- Surface a note on non-native quants so the estimate isn't silently optimistic/pessimistic
- Prefer scraper/runtime logic over hand-editing `hf_models.json` (per AGENTS.md)

Related: #969, #971

贡献指南

打开贡献指南

评估

这个 Issue 还没有评估数据。

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。