Comfy-Org / Comfy-Org/ComfyUI

Folder image list processing is inefficient for long batch workflows; consider image-by-image streaming execution

Open
#15,465 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
133k
Forks
15.7k
Avg merge
1d 10h
Merged PRs (30d)
153

Description

### Problem

Using the built-in **Load Image from Folder** node for a large set of images appears to execute the workflow in a list-oriented, node-by-node manner rather than letting each image run through the entire graph and be saved before moving to the next image.

For batch transformation workflows (for example, upscaling), this has several practical downsides:

- noticeably worse throughput than queueing images one at a time
- intermediate image data can stay alive longer than necessary, increasing RAM pressure
- output is not written as early as it could be
- if a downstream node errors late in the graph, a large amount of already-computed work may be lost and must be repeated

In one real workflow, processing about **150 images** via the folder loader took **more than 30 minutes**, substantially slower than expected compared with per-image execution.

### Desired behavior

For folder/dataset-style image processing, it would be useful to have an execution mode that behaves like a streaming `for each image` pipeline:

1. load one image
2. run that image through the full graph
3. save/output it
4. release intermediates where possible
5. continue with the next image

This would improve throughput, memory behavior, failure isolation, and resumability for large image-processing jobs.

### Request

Please consider either:

- changing the folder-loader execution path to support image-by-image streaming through the full graph, or
- adding an explicit execution mode/node for this behavior.

The current list semantics are useful in some workflows, but they are a poor fit for large batch conversion/upscaling jobs.

Contributor guide

Open the contributing guide

Research direction

Start by tracing the built-in Load Image from Folder node and its list-oriented execution path, then compare it with queueing images one at a time. Define how a streaming mode would load, process, save, and release one image before continuing, while preserving the current list semantics where needed. Done means large folder workflows improve throughput, memory behavior, failure isolation, and resumability.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
ai, backend
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.