Comfy-Org / Comfy-Org/ComfyUI

CLIP Vision Encode fails when the clip vision model is None

Open
#5,559 0 comments 0 reactions 0 assignees View on GitHub
Potential Bug
Dominant language
Python
Stars
133k
Forks
15.7k
Avg merge
1d 7h
Merged PRs (30d)
158

Description

### Expected Behavior

If the `clip_vision` input of the "CLIP Vision Encode" is `None` (e.g. an "unCLIPCheckpointLoader" node is used on a model without a clip vision embedding) then the `CLIP_VISION_OUTPUT` should be `None` as well.

### Actual Behavior

The workflow fails with a `NoneType` error when running/ealuating the "CLIP Vision Encode" node.

### Steps to Reproduce

1. Load the basic workflow.
2. Replace the "Load Checkpoint" node with an "unCLIPCheckpointLoader" node with a model that does not have a `clip_vision` model embedded (e.g. SD 2.1).
3. Hook up a "CLIP Vision Encode" node to the `clip_vision` output of (2).
4. Hook up an "unCLIPConditioning" node to a zero or prompt conditioning, and the output of (3).
5. Hook up the output of (4) to the positive KSampler input.

### Debug Logs

```powershell
n/a
```

### Other

The implementation of `CLIPVisionEncode` is:
```python
def encode(self, clip_vision, image):
output = clip_vision.encode_image(image)
return (output,)
```

This should be modified to:
```python
def encode(self, clip_vision, image):
if clip_vision is None:
return (None,)
output = clip_vision.encode_image(image)
return (output,)
```

Alternatively, if an `enabled` option is added it should be:
```python
def encode(self, clip_vision, image, enabled):
if not enabled:
return (None,)
output = clip_vision.encode_image(image)
return (output,)
```

Note: "unCLIPConditioning" can be disabled/bypassed by setting the strength to 0 so does not need to be modified.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.