ROCm / ROCm/AMDMIGraphX

MIGraphX EP places almost none of a conformer encoder (parakeet-tdt-0.6b-v3 ONNX) - CPU-speed inference

Open
#5,058 0 comments 0 reactions 1 assignee View on GitHub

@adityas-amd is already working on this.

Since Jul 13, 2026.

status: triage
Dominant language
C++
Stars
333
Forks
150
Avg merge
4d 19h
Merged PRs (30d)
54

Description

Environment

  • onnxruntime 1.24.4 ROCm build (Arch python-onnxruntime-opt-rocm), host ROCm 7.2.4
  • GPU: gfx1100 (RX 7900 XTX)
  • Model: NVIDIA parakeet-tdt-0.6b-v3 ONNX export (encoder-model.onnx, conformer
    with relative-position attention; the export istupakov/onnx-asr consumes)

Symptom

Session creation with ["MIGraphXExecutionProvider", "CPUExecutionProvider"]
"succeeds": a 2–4 s "Model Compile" (far too fast for a 0.6B encoder), then
inference runs at CPU speed with ~7 % GPU utilization and near-zero extra VRAM.
The encoder is evidently partitioned almost entirely to the CPU EP. Fixed-shape
add_free_dimension_override_by_name for all dynamic axes changes nothing, so
this is op support (likely the relative-position attention subgraph), not
dynamic shapes.

Ask

  1. Which ops/subgraph patterns in this encoder are unsupported? (Happy to run
    any diagnostic build/env-var and attach logs.)
  2. Conformer-style ASR encoders are a major model family — coverage here would
    make MIGraphX viable for a wide class of speech models.

Repro script + model layout available; the model is a public HF download.


Found while batch-transcribing ~650h of audio with parakeet-tdt-0.6b-v3 on an AMD gfx1100 workstation.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.