Comfy-Org / Comfy-Org/ComfyUI

More built-in data types: vector, matrix, color, image (with ColorSpace)

Open
#8,821 0 comments 0 reactions 0 assignees View on GitHub
Feature
Dominant language
Python
Stars
133k
Forks
15.7k
Avg merge
1d 6h
Merged PRs (30d)
155

Description

### Feature Idea

There are entire **categories** of nodes that are effectively impossible without some essential data types being pre-made. Sadly enough, most of them are about image processing in one way or another. Maybe, they should be added to the core? It shouldn't be hard to implement.

Here are some that come into mind first:

### 3-position toggle

https://github.com/comfyanonymous/ComfyUI/issues/9532

### Vectors/Matrices

https://github.com/comfyanonymous/ComfyUI/issues/9530

### Colors

https://github.com/comfyanonymous/ComfyUI/issues/9531

### Image

Similarly to Color type, it would be nice to have 2 sub-type specifiers for already existing `IMAGE`: whether it's values are expressed in sRGB or Linear colorspace:
- sRGB: 0.5 value visually looks like a middle gray.
- Linear: 0.5 looks like a bright gray, almost white. And yes, some might argue that there are many "linear" colorspaces, but let's stay away from this rabbit hole for now. Even this simple separation by common gamma-encoding would already be a huge improvement.

I propose to completely disallow images of an "unspecified" colorspace: it's either sRGB or Linear, no such thing as "just an image, I don't know what colorspace it's in - could be any".
And sRGB is assumed by default, so any image that's not marked as linear is treated like an sRGB one (almost 100% backwards compatibility with all the nodes here - only some extra-fancy linear-processing nodes would need an update, but that's ok).

Ideally, a few extra built-in methods should be added to convert between these two spaces:
- `get_as_is()`: always returns the image data as-is, with no conversion.
- `get_srgb()`: returns data as-is for sRGB-marked images, performs a linear->sRGB conversion for Linear-marked ones.
- `get_linear()`: the opposite.

For display in the browser, `get_srgb()` is used. So, even linear images would be displayed properly.

Unlike `IMAGE`, alpha is always treated as linear one (even in Photoshop), so `MASK` is good as is.

### Existing Solutions

- https://github.com/Amorano/Jovimetrix
- https://github.com/Amorano/Jovi_GLSL
- https://github.com/patriciogonzalezvivo/comfyui_glslnodes

### Other

_No response_

Contributor guide

Open the contributing guide

Research direction

No files, tests, or entry points are named. Start by locating the existing IMAGE and MASK data types and the browser display path, then determine how vectors, matrices, colors, color spaces, and the proposed conversion methods would fit the node system. Done would require an agreed scope and implementation for the selected built-in types, including sRGB/Linear behavior and compatibility with existing images.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
ai, computer-vision
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.