Comfy-Org / Comfy-Org/ComfyUI

Performance issue in `image_upload` / `compare_image_hash` (server.py)

Open
#12,965 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
133k
Forks
15.7k
Avg merge
1d 6h
Merged PRs (30d)
155

Description

I noticed an inefficiency in the duplicate-file detection logic implemented in `image_upload()` and `compare_image_hash()`.

### Current behavior

When uploading an image, `image_upload()` checks whether files with the same base name already exist. For each candidate filename (`image.png`, `image (1).png`, `image (2).png`, ...), it calls `compare_image_hash()` to determine whether the uploaded file is identical to an existing one.

`compare_image_hash()` currently:

1. Reads the entire existing file into memory.
2. Reads the entire uploaded file.
3. Computes hashes of both.

which is then repeated for the next candidate.

As a result:

- The uploaded file is re-read and re-hashed for every filename collision.
- Both files are fully read into memory rather than processed incrementally.
- The work scales with the number of name collisions.

### Observed inefficiencies

1. **No size check before hashing**

Files with different sizes cannot be identical. However, the current implementation still reads and hashes both files even when their sizes differ.

2. **Hashing is unnecessary for equality testing**

The function only needs to determine whether two files are identical. Hashing forces the entire contents of both files to be read and processed, even if they differ very early on (which is likely to be the case in practice). It would only make sense if you were to cache the hash values in memory. Otherwise you're just wasting CPU cycles.
A direct chunked byte comparison can terminate early as soon as a difference is found.

3. **Repeated full processing of the uploaded file**

Because the upload stream is rewound and reprocessed for each candidate file, the uploaded file may be read and hashed multiple times unnecessarily.

### Potential improvements

A more efficient approach would be:

1. Compare file sizes first.
2. If sizes differ --> files are different.
3. If sizes match --> compare files directly in chunks (e.g. 64 KB blocks).
4. Stop at the first mismatch.

### Expected impact

The current implementation performs well enough when few collisions exist, but performance degrades significantly when directories contain many files with the same base name (most pertinently when pasting images as `pasted/image.png`).

Optimizing this path should greatly improve performance during uploads in some cases. The current implementation can take several seconds to run if there are thousands of name collisions.

Contributor guide

Open the contributing guide

Research direction

Start in server.py at image_upload() and compare_image_hash(), and trace how the upload stream and candidate files are currently read. Compare file sizes first, then use chunked direct comparison with early exit; done means duplicate detection avoids repeated full hashing while preserving its existing behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend, performance
Issue type
Refactor
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.