microsoft / microsoft/onnxruntime-extensions

Vendored minja.hpp missing .replace() and other common Jinja string methods — chat templates from popular HF models fail to render

Open
#1,081 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

stale
Dominant language
C++
Stars
479
Forks
144
Avg merge
1d 8h
Merged PRs (30d)
11

Description

Summary

The vendored copy of minja at shared/api/minja.hpp is missing handlers for several standard Jinja string methods that current upstream minja supports. The most impactful gap is .replace(), which causes OrtxApplyChatTemplate to throw Unknown method: replace on chat templates shipped with widely-used Hugging Face models.

Downstream report: microsoft/Foundry-Local#800 (smollm3-3b chat template fails on system_message.replace("/no_think", "").replace("/think", "").rstrip()).

Reproduction

Apply any chat template that uses .replace() via OrtxApplyChatTemplate. Minimal example from smollm3-3b's chat_template.jinja:

{%- set custom_instructions = system_message.replace("/no_think", "").replace("/think", "").rstrip() -%}

Result:

Microsoft.ML.OnnxRuntimeGenAI.OnnxRuntimeGenAIException:
Unknown method: replace at row 23, column 48:
{%- set custom_instructions = system_message.replace("/no_think", "").replace("/think", "").rstrip() -%}
                                               ^

The error is thrown by the catch-all throw std::runtime_error("Unknown method: " + method->get_name()) at the end of the string-method dispatch in shared/api/minja.hpp.

Root cause

Diffing shared/api/minja.hpp against current upstream google/minja:

String method Upstream minja Vendored in ort-extensions
.strip / .rstrip / .lstrip
.split
.replace ❌ missing
.upper / .lower / .title ❌ missing

The vendored header declares itself as "based on the minja project, with modifications to be more VS and mobile compiler friendly", but appears to have drifted behind upstream on string-method coverage. These methods are not optional — they are standard Jinja2 string methods that Hugging Face's transformers chat templates rely on heavily.

Impact

Any model whose chat_template.jinja uses .replace(), .upper(), .lower(), or .title() will fail to render via OrtxApplyChatTemplate. This affects ORT GenAI and every downstream consumer (Foundry Local, custom apps using ORT GenAI's Tokenizer::ApplyChatTemplate).

Confirmed-affected model: smollm3-3b-generic-gpu:1. Likely also affects other recent HF model templates that use the same idioms (Qwen, Llama-3.x derivatives, several SmolLM variants).

Suggested fix

Two options, in preference order:

  1. Re-sync shared/api/minja.hpp with current upstream google/minja, preserving the VS / mobile compiler-friendly patches. This is the durable fix and would also pick up other upstream improvements since the last sync.
  2. Targeted patch adding handlers for .replace, .upper, .lower, .title (and any other string methods that have landed upstream since the last sync) into the existing string-method dispatch in shared/api/minja.hpp.

Either way, please also consider adding a regression test under test/pp_api_test/ that renders the chat templates of a handful of popular HF models (e.g. SmolLM3, Qwen3, Llama-3.x, Phi-4) against OrtxApplyChatTemplate to catch future drift between vendored minja and what HF model authors actually ship.

Cross-references

  • Downstream user report: microsoft/Foundry-Local#800
  • Related prior minja work in this repo: #1070 (OOB / parser recursion), #1001 (Qwen3 template), #965 (chat template impl refactor)

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in shared/api/minja.hpp at the string-method dispatch and compare the vendored implementation with current upstream google/minja, preserving the repository's compiler-friendly changes. Reproduce the failure through OrtxApplyChatTemplate using the shown SmolLM3 template, then inspect test/pp_api_test/ for regression coverage. Done means the listed methods render successfully and representative Hugging Face chat templates no longer fail.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
backend
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
64/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.