Comfy-Org / Comfy-Org/ComfyUI

Run ComfyUI on AWS Inferentia2

Open
#8,428 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
133k
Forks
15.7k
Avg merge
1d 6h
Merged PRs (30d)
155

Description

Hi,

AWS has a CPU-based instance type called **Inf2** ([link](https://aws.amazon.com/ec2/instance-types/inf2/)).

From [this](https://aws.amazon.com/it/blogs/machine-learning/maximize-stable-diffusion-performance-and-lower-inference-costs-with-aws-inferentia2/) article, they seem to perform almost as well as other GPU-based instances, but at a lower cost.

We did a quick test running a workflow on an Inf2 instance "as is", but of course is much slower than a GPU-based instance. A particular AWS SDK called [Neurons](https://awsdocs-neuron.readthedocs-hosted.com/en/latest/index.html) is required to compile models so they can run on Inf2 instances at maximum performance.

How ComfyUI can be run on this type of instance? Has anyone done this yet? Or is this a feature that will be included in the future releases of ComfyUI, maybe?

Thanks.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.