Chat templates are guessed from the model name instead of read from the model

RuvTokenizer::apply_chat_template does not use the model's chat template. It calls
ChatTemplate::detect_from_model_id, which lowercases the repo name and looks for substrings like
llama-3, qwen or gemma, then picks one of six hardcoded formats. Anything that matches nothing
falls through to ChatML.

The template the model actually ships is already on disk when this happens. from_pretrained
downloads tokenizer_config.json and passes it to load_special_tokens_from_config, so only the
special tokens are read. The chat_template field in that same file is never looked at.

I ran the templates of 20 models against what transformers.apply_chat_template produces for the
same messages. 57 cases. 47 came out different. Two models were right the whole way through,
Phi-3-mini and Qwen3.

Some concrete ones.

Hermes-3-Llama-3.1-8B. The name contains llama-3, so it gets the Llama 3 format. Hermes 3 is a
ChatML model. You get <|start_header_id|>user<|end_header_id|> where the model expects
<|im_start|>user.

DeepSeek-R1-Distill-Qwen-7B. The name contains qwen, so it gets ChatML. The real format is
<｜User｜> and <｜Assistant｜>, and the generation prompt ends with <think> and a newline. None
of that is produced.

zephyr-7b-beta gets ChatML. The real one is <|user|> with </s> after each turn.

granite-3.1-8b-instruct gets ChatML. The real one is <|start_of_role|> and it also builds a system
message with the knowledge cutoff and the date in it.

falcon-7b-instruct, command-r-v01, openchat-3.5 and deepseek-llm-7b-chat all match nothing and get
ChatML.

There is no way to work around this from outside. ChatTemplate::Custom(String) looks like an
escape hatch, but format_custom only does .replace() on {system}, {user} and {assistant}.
Passing a real jinja template into it gives you the jinja source back. It also joins every message
of the same role into one string, so multi turn does not survive.

Why this is easy to miss

The output always looks like a valid prompt. It is just not the one the model was trained on, so it
reads as the model being weaker than expected rather than as a bug. It also happens that Phi-3 and
Qwen are two of the formats that come out correct, and those are common models to test with.

The fix

Read chat_template out of tokenizer_config.json and render it. It is jinja, so it needs a jinja
engine and a few things transformers sets that are easy to miss, trim_blocks and lstrip_blocks,
the key order in tojson, and the raise_exception and strftime_now functions templates call.

I maintain hf-chat-template for exactly this.
It takes the template string and the special tokens and returns the prompt, and it checks its output
against python transformers on real hub templates. You already depend on serde and serde_json,
and the hub feature is off by default, so it does not touch your hf-hub version.

One thing to flag up front. Its MSRV is 1.85 and the workspace is on 1.77, so this would move that
floor.

I would like to send a PR that reads the real template and keeps detect_from_model_id as the
fallback for when a model ships no template.

Start at RuvTokenizer::apply_chat_template, then trace from_pretrained through load_special_tokens_from_config and inspect ChatTemplate::detect_from_model_id and format_custom. Compare the loaded tokenizer_config.json chat_template with transformers outputs, including multi-turn messages and generation prompts. Done means real Jinja templates render correctly, the model-name detector remains a fallback, and compatibility requirements such as trim_blocks, lstrip_blocks, tojson ordering, raise_exception, and strftime_now are covered.