More built-in data types: vector, matrix, color, image (with ColorSpace)
- Dominant language
- Python
- Stars
- 133k
- Forks
- 15.7k
- Avg merge
- 1d 6h
- Merged PRs (30d)
- 155
Description
### Feature Idea
There are entire **categories** of nodes that are effectively impossible without some essential data types being pre-made. Sadly enough, most of them are about image processing in one way or another. Maybe, they should be added to the core? It shouldn't be hard to implement.
Here are some that come into mind first:
### 3-position toggle
https://github.com/comfyanonymous/ComfyUI/issues/9532
### Vectors/Matrices
https://github.com/comfyanonymous/ComfyUI/issues/9530
### Colors
https://github.com/comfyanonymous/ComfyUI/issues/9531
### Image
Similarly to Color type, it would be nice to have 2 sub-type specifiers for already existing `IMAGE`: whether it's values are expressed in sRGB or Linear colorspace:
- sRGB: 0.5 value visually looks like a middle gray.
- Linear: 0.5 looks like a bright gray, almost white. And yes, some might argue that there are many "linear" colorspaces, but let's stay away from this rabbit hole for now. Even this simple separation by common gamma-encoding would already be a huge improvement.
I propose to completely disallow images of an "unspecified" colorspace: it's either sRGB or Linear, no such thing as "just an image, I don't know what colorspace it's in - could be any".
And sRGB is assumed by default, so any image that's not marked as linear is treated like an sRGB one (almost 100% backwards compatibility with all the nodes here - only some extra-fancy linear-processing nodes would need an update, but that's ok).
Ideally, a few extra built-in methods should be added to convert between these two spaces:
- `get_as_is()`: always returns the image data as-is, with no conversion.
- `get_srgb()`: returns data as-is for sRGB-marked images, performs a linear->sRGB conversion for Linear-marked ones.
- `get_linear()`: the opposite.
For display in the browser, `get_srgb()` is used. So, even linear images would be displayed properly.
Unlike `IMAGE`, alpha is always treated as linear one (even in Photoshop), so `MASK` is good as is.
### Existing Solutions
- https://github.com/Amorano/Jovimetrix
- https://github.com/Amorano/Jovi_GLSL
- https://github.com/patriciogonzalezvivo/comfyui_glslnodes
### Other
_No response_
Contributor guide
Research direction
No files, tests, or entry points are named. Start by locating the existing IMAGE and MASK data types and the browser display path, then determine how vectors, matrices, colors, color spaces, and the proposed conversion methods would fit the node system. Done would require an agreed scope and implementation for the selected built-in types, including sRGB/Linear behavior and compatibility with existing images.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- ai, computer-vision
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100