Comfy-Org / Comfy-Org/ComfyUI

Feature Request: Optional limited concurrency for API-based or lightweight workflows

Open
#12,082 5 comments 3 reactions 0 assignees View on GitHub
Feature
Dominant language
Python
Stars
133k
Forks
15.7k
Avg merge
1d 6h
Merged PRs (30d)
155

Description

### Feature Idea

First of all, thank you for ComfyUI. It has been extremely useful both as a local workflow tool and as an orchestration layer.

I understand and appreciate that ComfyUI currently uses a single global execution queue by default, prioritizing stability and avoiding GPU OOM issues. This design makes perfect sense for GPU-heavy local diffusion workflows.

However, I would like to propose an **optional and explicitly limited concurrency setting**, intended only for specific scenarios, such as:

- Workflows that primarily call **external image/video generation APIs**
- Nodes that are **network-bound or I/O-bound**
- Lightweight local workflows (e.g. small resolution images, low VRAM usage)

### Proposed idea (high-level)
- Introduce an optional setting such as `max_concurrent_jobs`
- Default value remains **1** (current behavior)
- Concurrency is **opt-in**, not enabled by default
- Clearly documented as **unsafe for heavy GPU workloads**
- Intended mainly for API-based or lightweight workflows

### Motivation
In API-based generation workflows, most of the heavy computation happens remotely. The local ComfyUI instance is often waiting for network responses, so strict serial execution underutilizes system resources and limits multi-user usage.

A controlled concurrency option could help ComfyUI better serve as:
- A workflow orchestrator
- A shared internal service for multiple users
- A front-end for remote generation pipelines

### Notes
- This request does **not** aim to turn ComfyUI into a general-purpose inference server
- Default behavior and stability should remain unchanged
- Users who enable this option fully accept the associated risks

Thank you for considering this request.

---

### 中文补充说明(Chinese version below)
以下是中文补充说明,供中文用户和社区参考:

我非常理解 ComfyUI 目前采用单一全局队列的设计,这在本地大模型、重 GPU 推理场景下是非常合理且稳定的做法。

这里提出的并发需求,并不是希望 ComfyUI 变成通用推理服务或高并发服务器,而是**在非常限定的场景下提供一个可选能力**,例如:

- 调用在线文生图 / 文生视频 API 的工作流
- 以网络等待或 I/O 为主的节点
- 本地跑小分辨率、低显存占用的轻量任务

核心想法是:
- 默认行为完全不变(并发 = 1)
- 只有用户明确开启时才允许有限并发
- 并清楚提示该功能不适用于重 GPU 推理场景,风险需用户自行承担

这样可以在不破坏 ComfyUI 稳定性的前提下,提升其在 API 编排、多用户共享等场景下的使用体验。

### Existing Solutions

I have searched existing custom nodes and extensions and did not find any solution that provides configurable execution concurrency at the core scheduler level.

Some API or HTTP call nodes exist, but they do not address global or workflow-level concurrency control.

### Other

_No response_

Contributor guide

Open the contributing guide

Research direction

No files or tests are identified. Start by tracing ComfyUI's global execution queue and core scheduler, then inspect how API-based workflows are submitted and tracked. Define how an opt-in concurrency limit would preserve the default of one job, document GPU-workload risks, and establish tests for the limit and default behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
ai, api, backend
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.