microsoft / microsoft/onnxruntime

[ROCM ep][fp16]: when to run stable-diffusion-xl, assert on rocm ep

Open
#19,291 3 comments 0 reactions 1 assignee View on GitHub

@cloudhan is already working on this.

Since Jan 27, 2024.

ep:ROCm
Dominant language
C++
Stars
21.9k
Forks
4.2k
Avg merge
4d 11h
Merged PRs (30d)
184

Description

### Describe the issue

assert info: "MultiHeadAttention of ROCm EP is only supported if tunable op is used and tuning is enabled".

### To reproduce

export onnx model as [the official guide](https://github.com/microsoft/onnxruntime/tree/main/onnxruntime/python/tools/transformers/models/stable_diffusion ):
`pip install optimum diffusers onnx onnxruntime-gpu
optimum-cli export onnx --model stabilityai/stable-diffusion-xl-base-1.0 --task stable-diffusion-xl ./sd_xl_base_onnx`
`python optimize_pipeline.py -i ./sd_xl_base_onnx -o ./sd_xl_base_fp16 --float16`

and then, run as:
`from optimum.onnxruntime import ORTStableDiffusionXLPipeline

pipeline = ORTStableDiffusionXLPipeline.from_pretrained('sd_xl_base_fp16', provider='ROCMExecutionProvider')
prompt = "a cat, cinematic photo cute pet . 35mm photograph, film, bokeh, professional, 4k, highly detailed"
image = pipeline(prompt=prompt, num_images_per_prompt=1, num_inference_steps=50, guidance_scale=0).images[0]

image.save("output_fp16.png")`

### Urgency

_No response_

### Platform

Linux

### OS Version

ubuntu18.04

### ONNX Runtime Installation

Built from Source

### ONNX Runtime Version or Commit ID

2b87dd373a3567c2c426e2f090b201b8b051a346

### ONNX Runtime API

Python

### Architecture

X64

### Execution Provider

Other / Unknown

### Execution Provider Library Version

rocm5.2

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.