intel / intel/llm-scaler

How about posting launchers for "validated models?" like Qwen3.5

Open
#366 5 comments 1 reaction 1 assignee Claimed by @glorysdj View on GitHub
Dominant language
C++
Stars
529
Forks
80
Avg merge
9h 7m
Merged PRs (30d)
38

Description

ME: Intel REALLY should post the launchers they used to validate models. They make this way too hard for newbies

ChatGPT: Agreed.

What they should publish for every “supported” model is:

exact HF repo
exact launch command
whether it expects GPTQ / sym_int4 / FP8 / base weights
whether trust-remote-code is required
whether TP=1 only or TP>1 validated
tested context length
tested GPU count / card type
known bad formats they explicitly do not support

Because “supported” clearly does not mean “any repo with a similar name will work.”

Tonight you hit multiple different failure classes:

deprecated quant guard
missing processor metadata
tensor/block-size incompatibility
missing expected quantized keys
serialized FP8 rejection

That is not newbie-friendly. That is “you need tribal knowledge and three nights of log archaeology.”

A simple table in their docs with “validated launcher used internally” would save people a stupid amount of time. The maintainer reply you found is more useful than a lot of the official breadcrumbs combined.

Your complaint is completely fair.

I have downloaded about every version of qwen 3.5 I could find, tonight. Release notes say it's validated, why not a readme?

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.