microsoft / microsoft/onnxruntime-extensions
Vendored minja.hpp missing .replace() and other common Jinja string methods — chat templates from popular HF models fail to render
Nobody has claimed this yet.
- Dominant language
- C++
- Stars
- 479
- Forks
- 144
- Avg merge
- 1d 8h
- Merged PRs (30d)
- 11
Description
Summary
The vendored copy of minja at shared/api/minja.hpp is missing handlers for several standard Jinja string methods that current upstream minja supports. The most impactful gap is .replace(), which causes OrtxApplyChatTemplate to throw Unknown method: replace on chat templates shipped with widely-used Hugging Face models.
Downstream report: microsoft/Foundry-Local#800 (smollm3-3b chat template fails on system_message.replace("/no_think", "").replace("/think", "").rstrip()).
Reproduction
Apply any chat template that uses .replace() via OrtxApplyChatTemplate. Minimal example from smollm3-3b's chat_template.jinja:
{%- set custom_instructions = system_message.replace("/no_think", "").replace("/think", "").rstrip() -%}
Result:
Microsoft.ML.OnnxRuntimeGenAI.OnnxRuntimeGenAIException:
Unknown method: replace at row 23, column 48:
{%- set custom_instructions = system_message.replace("/no_think", "").replace("/think", "").rstrip() -%}
^
The error is thrown by the catch-all throw std::runtime_error("Unknown method: " + method->get_name()) at the end of the string-method dispatch in shared/api/minja.hpp.
Root cause
Diffing shared/api/minja.hpp against current upstream google/minja:
| String method | Upstream minja | Vendored in ort-extensions |
|---|---|---|
.strip / .rstrip / .lstrip |
✅ | ✅ |
.split |
✅ | ✅ |
.replace |
✅ | ❌ missing |
.upper / .lower / .title |
✅ | ❌ missing |
The vendored header declares itself as "based on the minja project, with modifications to be more VS and mobile compiler friendly", but appears to have drifted behind upstream on string-method coverage. These methods are not optional — they are standard Jinja2 string methods that Hugging Face's transformers chat templates rely on heavily.
Impact
Any model whose chat_template.jinja uses .replace(), .upper(), .lower(), or .title() will fail to render via OrtxApplyChatTemplate. This affects ORT GenAI and every downstream consumer (Foundry Local, custom apps using ORT GenAI's Tokenizer::ApplyChatTemplate).
Confirmed-affected model: smollm3-3b-generic-gpu:1. Likely also affects other recent HF model templates that use the same idioms (Qwen, Llama-3.x derivatives, several SmolLM variants).
Suggested fix
Two options, in preference order:
- Re-sync
shared/api/minja.hppwith current upstreamgoogle/minja, preserving the VS / mobile compiler-friendly patches. This is the durable fix and would also pick up other upstream improvements since the last sync. - Targeted patch adding handlers for
.replace,.upper,.lower,.title(and any other string methods that have landed upstream since the last sync) into the existing string-method dispatch inshared/api/minja.hpp.
Either way, please also consider adding a regression test under test/pp_api_test/ that renders the chat templates of a handful of popular HF models (e.g. SmolLM3, Qwen3, Llama-3.x, Phi-4) against OrtxApplyChatTemplate to catch future drift between vendored minja and what HF model authors actually ship.
Cross-references
- Downstream user report: microsoft/Foundry-Local#800
- Related prior minja work in this repo: #1070 (OOB / parser recursion), #1001 (Qwen3 template), #965 (chat template impl refactor)
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in shared/api/minja.hpp at the string-method dispatch and compare the vendored implementation with current upstream google/minja, preserving the repository's compiler-friendly changes. Reproduce the failure through OrtxApplyChatTemplate using the shown SmolLM3 template, then inspect test/pp_api_test/ for regression coverage. Done means the listed methods render successfully and representative Hugging Face chat templates no longer fail.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- backend
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 64/100