[Feature][ONNX][MIGraphX EP] Offline pre-compilation, bundling, and batch-size-aware loading of `.mxr` artifacts for zero-stall deployment
@TedThemistokleous is already working on this.
Since Jul 22, 2026.
- Dominant language
- C++
- Stars
- 333
- Forks
- 150
- Avg merge
- 4d 19h
- Merged PRs (30d)
- 54
Description
DOR (Definition of Ready)
migraphx-driver compile --onnx --gpu --binary -oalready produces valid.mxr(msgpack) output, confirmed compatible withmigraphx::load()— the offline compilation primitive already exists- ORT MIGraphX EP already loads
.mxrfrom disk viapreload_mxr_cache_from_disk()(introduced in commit ROCm/onnxruntime@9c1485648f) - Current gap: the EP locates
.mxrfiles by an internal hash (make_hash()atmigraphx_execution_provider.cc, MurmurHash3 over input shapes) that external tooling cannot reproduce, and only from the globalORT_MIGRAPHX_MODEL_CACHE_PATH— so there is no externalized, batch-size-addressable contract for pre-built artifacts - Deployment stack: Triton Inference Server → ORT backend → MIGraphX EP, on AMD MI300X (
gfx942)
Description
As a model serving infra engineer deploying ONNX models on AMD MI300X via Triton + ORT MIGraphX EP,
I want to (1) pre-compile an ONNX model into per-batch-size .mxr artifacts in an offline environment,
(2) bundle and distribute those artifacts alongside the model, and (3) have the Triton ORT-backend
MIGraphX EP recognize and load the correct artifact by batch size at runtime through an externalized,
documented contract — so that inference never pays on-demand compilation stall time on the GPU host.
The offline compilation primitive already exists (migraphx-driver ... --binary), and the EP can
already load .mxr from disk. What is missing is the externalized contract that ties the three stages
together:
- Offline: a reproducible way to know which
.mxrfilename (or manifest entry) corresponds to a
given (model, batch size, precision) — today the mapping is an internal hash that cannot be
reproduced outside a liveInferenceSession. - Distribution: a way to ship pre-built, read-only artifacts as part of a model bundle, rather
than depending on the single global writableORT_MIGRAPHX_MODEL_CACHE_PATH. - Runtime: the EP loads the matching artifact by batch size from that contract, falling back to
on-demand compilation only on a miss.
DOD (Definition of Done)
Core requirement — an externalized, batch-size-addressable artifact contract:
- A reproducible, documented mapping from (model, batch size, precision flags) to its
.mxr
artifact — via a manifest file, a documented filename formula, or a CLI/Python helper — so
offline-built artifacts can be placed where the EP will find them without a live session - The Triton ORT-backend MIGraphX EP loads the matching
.mxrby batch size from this contract
at session init; on a miss it falls back to on-demand compilation and writes to
ORT_MIGRAPHX_MODEL_CACHE_PATHas today - End-to-end verified on MI300X: offline compile per batch size → bundle + distribute →
zero first-inference stall under Triton
Potential implementation (non-binding):
- A per-model precompiled artifact path, e.g. a
migraphx_precompiled_pathprovider option passed
through theparametersblock ofgpu_execution_acceleratorin Triton'sconfig.pbtxt,
resolved relative to the model version directory (./mxr→model_repository/<model>/1/mxr/),
keeping each model bundle self-contained and read-only-friendly
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.