Demo apps should not hardcode templates/model types
Open
@larryliu0820 is already working on this.
Since Dec 11, 2025.
- Dominant language
- Python
- Stars
- 5k
- Forks
- 1.2k
- Avg merge
- 2d 10h
- Merged PRs (30d)
- 581
Description
🚀 The feature, motivation and pitch
The demo apps currently hardcode the chat template and model types. This means they do not support models outside of a fixed list (e.g., qwen3, phi4, etc).
Moreover, model type is hardcoded with the name (sometimes incorrectly). For example, gemma3-270m is a text-only model and fails to run in the demo app because the demo app assumes all gemma3 models are multi-modal.
We should auto-detect model type and chat template in the demo apps.
- For HF models, use tokenizer.json to get the chat template.
- For non-HF models, allow users to provide a template.jinja.
- Somehow determine text-runner vs. multi-modal runner automatically
Alternatives
No response
Additional context
No response
RFC (Optional)
No response
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.