microsoft / microsoft/onnxruntime-genai

Config::overlay() on pipeline session_options causes "File doesn't exist" on Model load

Open
#2,004 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
C++
Stars
1.1k
Forks
354
Avg merge
2d 16h
Merged PRs (30d)
85

Description

Describe the bug
Config::overlay() appends pipeline entries instead of merging by model_id, causing "File doesn't exist" error on Model load

When calling Config.overlay() with a JSON that contains a model.decoder.pipeline array,
the overlay appends new pipeline entries instead of merging into existing entries by model_id.
This leaves the pipeline vector with extra entries that have empty filename fields,
causing og.Model(config) to fail with:

To Reproduce
Steps to reproduce the behavior:

Reproduction Steps

  1. Load a decoder-pipeline model config (e.g. one with embeddings, context, iterator, lm_head in the pipeline)
  2. Apply an overlay that targets pipeline sub-model session_options:
import onnxruntime_genai as og

config = og.Config("C:/path/to/model/")
overlay = '''{
  "model": {
    "decoder": {
      "pipeline": [
        {
          "submodel":  { "session_options": { "enable_profiling": "test" } }
        }
      ]
    }
  }
}'''
config.overlay(overlay)
model = og.Model(config)  # ← RuntimeError here

**Expected behavior**
The overlay should find the existing pipeline entry by model_id ("context", "iterator") and merge/update its fields in-place, consistent with how provider_options overlays work (matched by name).


**Additional context**
I guess the root cause is In src/config.cpp, PipelineModelObject_Element::OnObject() always calls emplace_back() without checking for an existing entry with the same model_id.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the Python reproduction in the issue, then inspect src/config.cpp and PipelineModelObject_Element::OnObject(). Compare its handling of pipeline entries with the provider_options overlay behavior described in the issue. Done means overlaying session_options updates the matching pipeline entry without adding entries with empty filename fields, and og.Model(config) loads successfully.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
backend
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.