AlexsJones / AlexsJones/llmfit
fix(fit): prefer native MXFP4 quant path for gpt-oss-class MoE models
- 主要语言
- Rust
- 星标
- 36.3k
- 派生
- 2.3k
- 平均合并
- 2 天 18 小时
- 30 天内合并 PR
- 85
描述
## Context
PR #971 makes `plan --quant Q4_K_M --profile ryzen-ai-max-plus-395 openai/gpt-oss-120b` report **~50 tok/s** (matching the #969 measured reference). Dynamic `fit` still picks **Q8_0** on a 128 GB profile and estimates **~23.5 tok/s**, because gpt-oss ships **MXFP4-native** weights and the Tier-1 scalable term prices them at `quant_bpp("Q8_0")`.
## Ask
- Teach quant selection / bpp that `gpt_oss` (and similar) prefer native MXFP4 / low-precision quants when available
- Surface a note on non-native quants so the estimate isn't silently optimistic/pessimistic
- Prefer scraper/runtime logic over hand-editing `hf_models.json` (per AGENTS.md)
Related: #969, #971
贡献指南
评估
这个 Issue 还没有评估数据。